Institute of 
Education Sciences 


NCES 2021-029 
U.S. DEPARTMENT OF EDUCATION 


IES RZ Institute of 
am Education Sciences 


2012-2016 Program for International Student 
Assessment Young Adult Follow-up Study (PISA YAFS) 


How reading and mathematics performance at age 15 relate to 
literacy and numeracy skills and education, workforce, and life 
outcomes at age 19 


Research and Development Report 


JUNE 2021 


Saida Mamedova 

Maria Stephens 

Yuqi Liao 

Josh Sennett 

Paul Sirma 

American Institutes for Research 


Samantha S. Burg 
Project Officer 
National Center for Education Statistics 


NCES 2021-029 
U.S. DEPARTMENT OF EDUCATION 


U.S. Department of Education 
Miguel Cardona 
Secretary 


Institute of Education Sciences 
Mark Schneider 
Director 


National Center for Education Statistics 
James L. Woodworth 
Commissioner 


The National Center for Education Statistics (NCES) is the primary federal entity for collecting, analyzing, and 
reporting data related to education in the United States and other nations. It fulfills a congressional mandate to 
collect, collate, analyze, and report full and complete statistics on the condition of education in the United States; 
conduct and publish reports and specialized analyses of the meaning and significance of such statistics; assist state 
and local education agencies in improving their statistical systems; and review and report on education activities 
in foreign countries. 


NCES activities are designed to address high-priority education data needs; provide consistent, reliable, complete, 
and accurate indicators of education status and trends; and report timely, useful, and high-quality data to the 

U.S. Department of Education, the Congress, the states, other education policymakers, practitioners, data users, 
and the general public. Unless specifically noted, all information contained herein is in the public domain. 


We strive to make our products available in a variety of formats and in language that is appropriate to a variety of 
audiences. You, as our customer, are the best judge of our success in communicating information effectively. If you 
have any comments or suggestions about this or any other NCES product or report, we would like to hear from 
you. Please direct your comments to 


NCES, IES, U.S. Department of Education 
Potomac Center Plaza 

550 12th Street SW 

Washington, DC 20202 


June 2021 


The NCES Home Page address is https://nces.ed.gov. 
The NCES Publications and Products address is https://nces.ed.gov/pubsearch. 


This publication is only available online. To download, view, and print the report as a PDF file, go to the NCES 
Publications and Products address shown above. 


This report was prepared for the National Center for Education Statistics under Contract No. ED-IES-12-D-0002 
with American Institutes for Research. Mention of trade names, commercial products, or organizations does not 
imply endorsement by the U.S. government. 


Suggested Citation 

Mamedova, S., Stephens, M., Liao, Y., Sennett, J., and Sirma, P. (2021). 2012-2016 Program for International Student 
Assessment Young Adult Follow-up Study (PISA YAFS): How reading and mathematics performance at age 15 relate to 
literacy and numeracy skills and education, workforce, and life outcomes at age 19 (NCES 2021-029). U.S. Department 
of Education, National Center for Education Statistics. Washington, DC. Retrieved [date] from https://nces.ed.gov/ 
pubsearch/pubsinfo.asp?pubid=2021029. 


Content Contact 
Samantha S. Burg 

(202) 245-7537 
samantha.burg@ed.gov 


Acknowledgments 


The authors wish to thank the young adults that participated in PISA YAFS. Without their cooperation and 
willingness to share information about themselves, this study would not be possible. The authors also wish 
to thank all those who contributed to the PISA YAFS design, implementation, and data collection as well as 
the writing, production, and review of this report. The authors would also like to thank Patrick Gonzales, 
a former project officer for PISA at the National Center for Education Statistics (NCES), who oversaw the 
PISA YAFS study from its inception to May 2020, when he retired from NCES. 


ili 


Executive Summary 


The Program for International Student Assessment 
Young Adult Follow-up Study (PISA YAFS) is a new study 
that allows research into the characteristics, cognitive 
skills, and other life outcomes of young adults as they 
transition from high school to postsecondary life. This 
Research and Development report provides a snapshot 
of outcomes at the outset of this important transition, as 
well as preliminary examinations of how these outcomes 
are related to individuals’ competencies in high school. It 
is intended to serve as an example of using multiple data 
sources in longitudinal research and to contribute to the 
literature on education and employment pathways. 


Study design 


PISA YAFS was conducted in the United States in 2016 
with a sample of young adults (at age 19) who participated 
in PISA 2012 when they were in high school (at age 15). In 
PISA YAFS, students took the Education and Skills Online 


Table A. 


Time frame of data collection 


(ESO) literacy and numeracy assessments, which are 
based on the Program for the International Assessment of 
Adult Competencies (PIAAC). 


Both PISA and PIAAC, on which ESO is based, were 
designed to meet the overarching goal to “identify and 
develop the knowledge and skills that drive better jobs 
and better lives, generate prosperity and promote social 
inclusion” (OECD n.d.). As such, both assessments seek 
to measure key competencies that are applicable across 
a wide range of culturally and linguistically diverse 
countries, recognize a continuum of competency, 

and reflect real-world demands and contexts. While 
each study has its own unique framework, there are 
similarities in the content and skills measured and the 
overall approach and philosophy between the two. 
Table A provides a brief overview of the assessment 
components of PISA 2012 and ESO. Each also included 
a background questionnaire for participants. 


Overview of assessment data sources in PISA YAFS: 2012 and 2016 
ad beYAW/A Ih 
October—November 2012 


RSTO (ey-FX-Ye Meo} a Wl ad P-VANC9) 
March—July 2016 


Age at data collection 


15 years old 


19 years old’ 


Subjects included for PISA YAFS 


Reading and Mathematics Literacy? 


Literacy and Numeracy 


Mode Computer-based, adaptive 


100-400 points 


Paper-and-pencil 


Effective scales (for each subject)® 200-800 points 


Low (Below level 2) 
Middle (Levels 2-4) 
High (Level 5 or above) 


Low (Below level 2) 
Middle (Levels 2-3) 
High (Level 4 or above) 


Proficiency-level categories* 


' Estimates for 19-year-olds are for individuals who were 15-year-old students in fall 2012 and who participated in PISA YAFS in 2016. 

2 To avoid confusion with the ESO literacy and numeracy assessments, PISA subjects are referred to as “reading” and “mathematics” when reporting the 
results. 

3 Effective scales are estimated using three standard deviations up and down from the mean, as determined for each program. Score distributions show 
that few students score outside these ranges (OECD 2020; Woodworth 2019). 

4 Both the individual and categorized proficiency levels have their own assessment-specific definitions. Because the groupings were based on the same 
rationales for each assessment, they are similar enough for general comparison but should not be considered fully equivalent. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Highlights 
Because of its design, PISA YAFS is able to provide data 
on literacy, numeracy, and life outcomes at age 19, as 
well as their relationship to individuals’ reading and 
mathematics literacy at age 15. This summary focuses 
on those outcomes and relationships and explores 
whether they differ based on individuals’ background 
characteristics or the socioeconomic status of the schools 
they attended at age 15. 


Results are described mainly by proficiency levels, 
which are based on descriptions of the types of tasks 
that individuals at the various levels have demonstrated 
they can perform. For reporting, the levels for each 
assessment have been grouped into low, middle, and 
high levels (see table A). 


Proficiency in ESO at age 19 


© In 2016, in both subjects, most 19-year-olds 
performed at the middle level of proficiency in ESO: 
72 percent in literacy and 68 percent in numeracy 
(see figure A). About one-fifth to one-quarter 


Figure A. U.S. 19-year-olds’ proficiency in ESO, by 
subject and level: 2016 
Percent 
100 


80 
60 
40 


20 


ESO literacy 


ESO numeracy 


i High (Level 4 and above) 
Middle (Levels 2-3) 
Bf Low (Below level 2) 


NOTE: See appendix B for additional detail. Detail may not sum to totals 
due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education 
Statistics, Program for International Student Assessment Young Adult 
Follow-up Study (PISA YAFS), 2016. 


performed at the low level, and small percentages 
(10 percent or less) performed at the high level. 


© = The distribution across ESO proficiency levels was not 
statistically different in either subject by sex, language 
spoken at home, or whether a student was born in 
the United States, but it did differ by race/ethnicity. 
Proportionately more Black 19-year-olds were in the 
low level of proficiency in ESO numeracy compared 
with the population overall (45 vs. 25 percent, 
respectively). 


Relationship of proficiency in ESO at 
age 19 to proficiency in PISA at age 15 


Proficiency at age 19 was most closely related to 
proficiency at age 15 among the middle performers, who 
were the largest group in the PISA YAFS population at 
age 15 as well (77 percent in reading and 68 percent in 
mathematics) (see table B). 


© Of the U.S. 19-year-olds who performed at the middle 
level of proficiency in PISA 2012 reading at age 15, 
about 78 percent also performed at the middle level 
of proficiency in ESO literacy at age 19. Seven percent 
of these PISA middle performers were at the high 
level of proficiency in ESO literacy at age 19, and 
15 percent were at the low level. 


© Of the 19-year-olds who performed at middle level of 
proficiency in PISA 2012 mathematics at age 15, about 
79 percent also performed at the middle level of 
proficiency in ESO numeracy at age 19. Four percent 
of these PISA middle performers were at the high 
level of proficiency in ESO numeracy at age 19, and 
17 percent were at the low level. 


The pattern for 19-year-olds who were high and low 
performers in PISA 2012 was not as stark: They were 

not as consistently in the same-labeled proficiency- 

level categories as they were at age 15. In fact, among 

the PISA 2012 high performers, at least half were in a 
different category in ESO at age 19—mainly in the middle 
proficiency level (57 percent in literacy and 54 percent 
in numeracy). Additionally, 49 percent of PISA 2012 low 
performers in literacy and 40 percent in numeracy were, 
by age 19, in the ESO middle proficiency level. In the case 
of both high and low performers, however, the overall 
percentages in these PISA 2012 categories were relatively 
small (22 percent or less). 


When examined separately for different subgroups, 

the pattern of a predominant and stable middle group 
held true, regardless of 19-year-olds’ background 
characteristics such as sex and race/ethnicity. However, 


Table B. 


Percent of PISA YAFS 
population who were 
PISA 2012 middle 
red=Yu coy at at-Y ecm (-\° [-WM Bo) 


PISA 2012 subject 
Reading 77 


Percentage of PISA 2012 middle performers, by ESO proficiency levels: 2012 and 2016 


Percent of PISA 2012 middle performers, =i) 
by ESO proficiency levels (age 19) 


High (Level 4 Middle Low 
lave mr: |ofe)"-9)] ((RW(-1 [72 )) (Below level 2) 


7 78 15 


Mathematics 68 


4 79 17 


NOTE: See appendix B for additional detail. Detail may not sum to totals due to rounding. Results for middle performers (levels 2-4) in PISA 2012 
reading are distributed across proficiency levels in ESO literacy and those in PISA 2012 mathematics across proficiency levels in ESO numeracy. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


the pattern diverged for 19-year-olds who, at age 15, were 
in schools with high percentages of students eligible for 
free or reduced-price lunch (FRPL).! 


© Inboth subjects, proportionately more PISA 2012 
middle performers who had been in the most 
economically challenged schools (i.e., those with 
75 percent or more of students receiving FRPL) 
scored at the low level of proficiency in ESO in 
2016 than did the population overall (28 vs. 

15 percent in literacy and 36 vs. 17 percent in 
numeracy, respectively). 


© In mathematics, proportionately fewer of the PISA 
2012 middle performers who had been in the most 
economically challenged schools scored at the middle 
level of proficiency in ESO numeracy than did the 
population overall (62 vs. 79 percent, respectively). 


Correlations between ESO scores at age 
19 and PISA 2012 scores at age 15 


Correlation describes the strength of a relationship 
between two measures. In this report, analyses examine 
the correlation of ESO and PISA scores—or the degree to 
which performance at age 19 was associated with that at 
age 15. 


© USS. 19-year-olds with higher literacy and numeracy 
scores in ESO tended to have had higher reading and 
mathematics scores in PISA 2012, respectively, at age 
15. The correlation between ESO literacy and PISA 
2012 reading is 0.59 and between ESO numeracy 
and PISA 2012 mathematics is 0.69, indicating strong 
positive relationships.” 


© Inboth subjects, however, correlations were weaker 
for 19-year-olds who at age 15 had attended the most 
or the least economically challenged schools (i.e., 


those with more than 75 percent or less than 
25 percent of students receiving FRPL) than for 
the PISA YAFS population overall. 


Relationship of education, workforce, 
and life outcomes at age 19 to 
proficiency in PISA at age 15 


The report also examined 19-year-olds’ education 
outcomes (degrees currently pursued, area of study 
currently pursued, and participation in nonformal 
education); workforce outcomes (employment status, 
combined employment and education status, and 
current occupation); and life outcomes (self-efficacy, 
life satisfaction, and vocational interests) in 2016 in 
relationship to their proficiency in PISA 2012 at age 15. 
Significant relationships were found for five of the nine 
outcome areas examined in the study, including the 
following: 


© High reading and mathematics proficiency at age 15 
were associated with higher education trajectories 
and other potentially advantageous life outcomes 
at age 19. The following examples are for high 
performers in reading, but the patterns apply to 
high performers in mathematics as well. 


= High performers in PISA 2012 reading were 
enrolled in bachelor’s degree programs at a 
higher rate (84 percent) than the population 
overall (45 percent) at age 19. 


= About 27 percent of high performers in PISA 2012 
reading were pursuing a postsecondary degree in 
sciences and 15 percent in engineering at age 19, 
compared with 12 and 7 percent, respectively, in 
the population overall. Science and engineering 
are two fields that policymakers and researchers 


‘The percentage of students eligible for free or reduced-price lunch was available for public schools only. 
?Cohen’s (1988) convention is used for interpreting correlation effect sizes. For more information, see section 2 of the report. 


generally consider critical to meeting the 
workplace and problem-solving demands of 
the future (U.S. Department of Education 2018, 
2020). 


© Low reading and mathematics proficiency at age 


15 was associated with lower education trajectories 
at age 19. The following examples are for low 
performers in reading, but the patterns apply to low 
performers in mathematics as well. 


= Low performers in PISA 2012 reading (i.e., 
below level 2) were more commonly still in high 
school (23 percent) than the population overall 


(9 percent) at age 19. 


Low performers in PISA 2012 reading were 

less commonly enrolled in bachelor’s degree 
programs (12 percent) than the population overall 
(45 percent) at this age. 


Patterns of vocational interest at age 19 also differed 
based on proficiency in PISA 2012 at age 15. The 
following examples are for high performers in 
reading, but the patterns apply to high performers in 
mathematics as well. 


= High performers in PISA 2012 reading had 

a higher level of interest in investigative 
vocations than the population overall at age 19 
(scoring 20 vs. 18, respectively, out of a total of 
AO points).? Investigative vocations include work 
that involves ideas and thinking rather than 


physical activity or leading people. 


High performers in PISA 2012 reading had a lower 
level of interest in enterprising vocations than 

the population overall at age 19 (scoring 17 vs. 

19, respectively). Enterprising vocations include 
work that involves starting up and carrying out 
business projects. 


Together, the results from PISA YAFS provide a snapshot 
of the cognitive skills of U.S. 19-year-olds as they move 
from high school to postsecondary life and an analysis 

of how their skills, outcomes, attitudes, and interests 

at that age are related to the academic proficiency they 
demonstrated at age 15. By documenting generally strong, 
positive relationships between individuals’ reading and 
mathematics performance at age 15 and their literacy 
skills, numeracy skills, and educational trajectories 

4 years later, this report provides an indication of the 
degree to which success on PISA 2012 is related to various 
outcomes at this important transitional stage and the 
outset of adult life. 


3 The results on vocational interest come from 19-year-olds’ responses to questions about the degree to which they would like or dislike various 
activities across six dimensions originally identified in the Holland (1997) RIASEC model: realistic, investigative, artistic, social, enterprising, and 
conventional. The questions were on a 5-point scale, and individuals’ responses were added for each dimension and ranged from 0 to 40. The higher 
the score, the higher the interest in the given type of vocation. See appendix B for additional details. 
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Foreword 


The Research and Development series of reports at the National Center for Education Statistics (NCES) 
has been initiated to 


© share studies and research that are developmental in nature. The results of such studies may be 
revised as the work continues and additional data become available; 


© share the results of studies that are, to some extent, on the cutting edge of methodological 
developments. Emerging analytical approaches and new computer software development often 
permit new and sometimes controversial analyses to be done. By participating in frontier research, 
we hope to contribute to the resolution of issues and improved analysis; and 


© participate in discussions of emerging issues of interest to education researchers, statisticians, and 
the federal statistical community in general. Such reports may document workshops and symposia 
sponsored by NCES that address methodological and analytical issues or may share and discuss 
issues regarding NCES practices, procedures, and standards. 


The common theme in all three goals is that these reports present results or discussions that do not 
reach definitive conclusions at this point in time, because the data are tentative, the methodology is 
new and developing, or the topic is one on which there are divergent views. Therefore, the techniques 
and inferences made from the data are tentative and subject to revision. To facilitate the process of 
closure on the issues, we invite comment, criticism, and alternatives to what we have done. Such 
responses should be directed to 


Marilyn Seastrom, Chief Statistician 

Director, Statistical Standards and Data Confidentiality Staff 
National Center for Education Statistics 

Institute of Education Sciences 

Potomac Center Plaza (PCP) 

550 12th Street SW 

Washington, DC 20202 
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1. Introduction 


Overview of PISA YAFS 


The Program for International Student Assessment 
Young Adult Follow-up Study (PISA YAFS) is a new study 
that uses international assessment data to examine the 
characteristics and outcomes of U.S. young adults at 

a key stage in their development-the transition from 
high school to their postsecondary lives. It provides 

a snapshot of their outcomes in 2016 at about age 19 
and examines how those outcomes are related to the 
knowledge and skills they demonstrated at age 15. The 
study is coordinated by the National Center for Education 
Statistics (NCES). 


For data on 19-year-olds, PISA YAFS used the Education 
and Skills Online (ESO) survey, which is based on 

the Organization for Economic Cooperation and 
Development’s (OECD) Program for International 
Assessment of Adult Competencies (PIAAC). ESO is a 
streamlined version of PIAAC, and both ESO and PIAAC 
include assessments in literacy, numeracy, and problem 
solving in technology-rich environments that were 
developed to measure the skills of people ages 16 to 65. 
In 2016, PISA YAFS also included a questionnaire about 
education, employment, and other characteristics. 


For data on 15-year-olds, the study relied on the 2012 
administration of the Program for International Student 
Assessment (PISA), which is the OECD’s flagship student 
assessment program that, every 3 years, assesses reading, 
mathematics, and science literacy and collects relevant 
background data on students and their schools. The 
PISA YAFS design involved recruiting students who had 
participated in PISA in 2012, when they were about 

15 years old, to take the ESO literacy and numeracy 
assessments 4 years later, in the spring and summer of 
2016, when they were about 19 years old.! 


This Research and Development report describes the 
literacy and numeracy performance of 19-year-olds 


along with other outcomes such as their engagement in 
postsecondary education, participation in the workforce, 
attitudes toward their lives and ability to make their 
own choices, and vocational interests. Moreover, it 
examines whether and how these outcomes are related 
to the 19-year-olds’ earlier performance in PISA 2012 and 
whether those relationships differ for various subgroups 
of individuals. This report thus provides an indication 
of the degree to which success on PISA 2012 is related 

to various outcomes at the important transition from 
high school to postsecondary life, and it extends NCES’s 
long tradition of longitudinal research on postsecondary 
outcomes (which will be described in the next section 
along with other literature on youth transitions). 


Importance of youth 
transitions 


PISA YAFS is part of a body of national and international 
research on youth transitions. Youth transitions are the 
key stages of development that occur when youth move 
from one school level to another (or when they leave 
school), and they tend to represent shared experiences. 
Commonly studied youth transitions include those 

that occur at the beginning of adolescence and at the 
beginning of young adulthood, as in PISA YAFS. 


The transition into young adulthood has received 
attention in recent years, particularly because of well- 
documented cultural and economic shifts that have 
affected that transition. Over the past two decades, there 
has been an upward trend in postsecondary enrollment 
and attainment, which has delayed workforce entry 

for some young adults (Chen et al. 2017; Furstenberg 
2010). At the same time the costs of that education 

have risen dramatically (Horn and Paslov 2014). This 

has left increasing percentages of individuals with 
substantial amounts of college debt that, together with an 
uncertain economy, could dampen their pursuit of future 


‘Literacy and numeracy were selected from among ESO’s domains for PISA YAFS because these are the subjects ESO and PISA 2012 have in common 


and can be examined over time. 


educational, employment, or family goals (College Board 
2017; Settersen and Ray 2010; Woo and Shaw 2015). Other 
traditional aspects of the transition to young adulthood, 
such as independent living, marriage, and building a 
family, can also be increasingly delayed (Furstenberg 
2010; Settersen and Ray 2010). It is important to 
understand the competencies young adults have and 

to know the choices they are making at this important 
stage—and whether those competencies and choices are 
related to any antecedent factors. 


Several recent international studies have followed-— 

as PISA YAFS does-the 15-year-old students who 
participated in PISA to determine how their performance 
in PISA is related to their outcomes as young adults. 

At least seven countries have undertaken such studies 
(Borgonovi et al. 2017). In some cases (e.g., Australia, 
Switzerland, and Uruguay), these were longitudinal 
studies and they either linked PISA students to ongoing 
administrative records or conducted periodic follow- 

up surveys (which did not have skills assessment 
components) with the students. In Denmark, the Ministry 
of Education administered the PIAAC assessment to a 
sample of students 12 years after they had initially taken 
PISA 2000; in Canada, the PISA reading assessment was 
readministered to a sample of PISA 2000 students 9 years 
after it was initially administered to them as 15-year- 

olds. As mentioned, the PISA YAFS study is designed in a 
similar vein as these studies, albeit generally on a shorter 
time horizon and without the multiple follow-ups that 
some of them have. 


These international studies generally found strong 
associations between PISA performance and participants’ 
outcomes as young adults, validating PISA as a potential 
predictor of students’ future success. For example, 

in Switzerland, PISA 2000 reading performance was 
positively associated with high school completion and 
college entry, and in Uruguay, PISA 2003 and 2006 
mathematics performance was positively associated 
with completing high school and negatively associated 
with dropping out of school (Borgonovi et al. 2017). In 
Canada, poor PISA 2000 performance was associated 
with a higher risk of poor labor market outcomes 

and lower uptake of postsecondary education (OECD 
2010a). In Denmark, the nature of the study allowed it 
to examine the link between competencies across time 
(Rosdahl 2014). This study showed that that the higher 
the PISA 2000 reading score, the higher the probability 


the participant was in the top third of performers on 

the PIAAC literacy scale. However, this study also noted 
certain mitigating factors that were associated with either 
an increase or decrease in competencies. For example, 
those participants whose parents were better educated, 
who themselves had received further education, or who 
had not had disruptions (e.g., illness or unemployment) 
that necessitated social welfare payments increased their 
relative competency level in PIAAC compared to those at 
similar initial competency levels in PISA 2000. Increased 
learning opportunities were also found to mitigate the 
relationship between higher socioeconomic status and 
stronger performance in the study in Australia (Borgonovi 
et al. 2017). Similarly, a cross-sectional study that 
compared PISA 2000 students with their comparable age 
cohort in PIAAC 2012 suggested that the negative effects 
of socioeconomic status on the decline of competencies 
was stronger for the lowest performers than for the 
highest performers (OECD 2017).? 


Each of these forgoing examples shared PISA as the 

lens through which youth transitions were examined, 
and each showed how international data can be used 
nationally, without a comparative aspect, to answer 
research questions of interest. In this way, they seem to 
have the most in common with PISA YAFS. However, PISA 
YAFS also builds from and extends the research on youth 
transitions that have used national data sources. National 
research includes the numerous studies that have been 
undertaken by NCES (which sponsored PISA YAFS) and 
the National Assessment Governing Board (NAGB), which 
aim to identify specific factors that are predictive of 
successful outcomes for young adults or preparedness for 
their postsecondary paths.? 


NCES research on the topic can be categorized into three 
groups of studies: (1) longitudinal studies that follow 

U.S. students in and beyond their educational careers, 
(2) transcript studies that examine the relationship 
between coursetaking patterns and outcomes, and 

(3) other cross-survey studies, such as PISA YAFS. 
Examples of longitudinal studies include High School and 
Beyond, the National Education Longitudinal Study of 
1988, the Education Longitudinal Study of 2002, and the 
High School Longitudinal Study of 2009. These studies 
have demonstrated, for example, that mathematics 
coursetaking and GPA in high school are strong 
predictors of students’ enrollment in postsecondary 
education (Kena et al. 2016). Similarly, the Beginning 


?TIn the referenced OECD study, socioeconomic status was measured by parental educational attainment and number of books in the home at 15 to 


16 years of age. 


3NCES also conducts longitudinal studies at other key transition points, such as early childhood (https://nces.ed.gov/ecls/) and the middle grades 


(https://nces.ed.gov/surveys/mgls/). 


Postsecondary Students Longitudinal Study found, at 
the 3-year check-in with 2011 first-time postsecondary 
students, a significant relationship between mathematics 
coursetaking and enrollment in 4-year colleges (Ifill 

et al. 2016). Diving deeper on coursetaking, the High 
School Transcript Study showed that, in 2009, high 
school graduates who completed a rigorous curriculum, 
an Advanced Placement or International Baccalaureate 
course, or another higher-level course in ninth grade 
had higher average National Assessment of Educational 
Progress (NAEP) scores compared with those who had 


completed a mid-level or standard curriculum (Nord et al. 


2011). 


Within NAGB, research on the transition from high 
school has focused on exploring whether the NAEP 
grade 12 assessments can be validated as indicators 

of preparedness for either academic or job training 
endeavors. For example, Fields (2014) explored 
identifying postsecondary preparedness indicators that 
include both education and career pathways, utilizing 
NAEP and other existing data sources. 


The results of the PISA YAFS study extend this body of 
international and domestic research that has followed 
students over time to better understand what predicts 
and supports their success as young adults. 


Organization of the report 


The next sections of this report are organized as follows: 


© Section 2: Study methods, including information 
on data sources and measures, reporting of results, 
sample, and data limitations 


© Section 3: Findings on performance in ESO and its 
relationship to PISA 2012 


© Section 4: Findings on the transition from high school 
to postsecondary life 


The report concludes with a brief summary of the 
findings. 


Exhibits and figures are included in the main body of the 
report, while all data tables are provided in appendix A. 
Appendix B describes the variables used in analyses, and 
appendix C provides brief technical notes. 


A full technical report will be available following 

the publication of this current report. It will include 
additional details on the data collection instruments, 
response rates, and nonresponse bias, among other 
topics. 


Where to find more information 


More information on the assessments described in this report can be found at the links below. 


© Program for International Student Assessment (PISA) 2012-—the assessment of 15-year- 
olds and pool for the PISA YAFS participants in 2016 


International website 


National website 


© Education and Skills Online (ESO)—the assessment tool used to collect data from 19-year-olds 


in PISA YAFS in 2016 


© Program for the International Assessment of Adult Competencies (PIAAC)-the study 
on which ESO’s scales and proficiency levels are based 


International website 


National website 


PIAAC Gateway 


For general information on PISA YAFS, see the PISA YAFS study website. For additional 
information, reports, and data tools on these and other international assessments, see the NCES 


international assessments website. 


2. Study Methods 


Data sources and measures 


PISA YAFS used two assessment tools to make longitudinal 
comparisons. Participants were first assessed in PISA 2012 
at age 15 and later assessed in ESO at age 19. This section 
thus addresses the content and skills in which PISA YAFS 
young adults were assessed at ages 15 and 19 as well as 

the background information collected about them in each 
assessment. 


Content and skills measured by 
PISA 2012 at age 15 


PISA 2012 was designed to meet the OECD Education 
Directorate’s overarching goal to “identify and develop 
the knowledge and skills that drive better jobs and better 
lives, generate prosperity and promote social inclusion” 
(OECD n.d.). The PISA 2012 framework specifies three 
main content domains—reading literacy, mathematics 
literacy, and scientific literacy—and two additional 
domains—problem solving and financial literacy. Only 
reading and mathematics literacy were used in PISA 
YAFS. In 2012, PISA was administered as a paper-and- 
pencil assessment. 


PISA 2012 reading literacy assessment 

In PISA 2012, reading literacy was defined as 
“understanding, using, reflecting on, and engaging with 
written texts in order to achieve one’s goals, develop 
one’s knowledge and potential, and participate in 
society” (OECD 2013b, p. 2). This was measured by 
providing respondents with a set of texts about which 
they answered questions or performed related tasks in 
various contexts. 


In terms of texts, PISA includes continuous text (which is 
prose text made up of sentences formed into paragraphs, 
such as in newspaper and magazine articles, brochures, 
manuals, e-mails, and many web pages) as well as 
noncontinuous text (which uses explicit typographic 
features, rather than paragraphs, to organize information 
into a matrix, such as in tables, graphs, charts, and 


forms). Additionally, some texts in PISA 2012 include 
elements of both continuous and noncontinuous text 
(mixed text), and some present multiple texts. 


In terms of the cognitive skills required by the questions 
and tasks in PISA 2012, these can be categorized into 
three broad strategies that are necessary for achieving a 
full understanding of texts: 


© Accessing and identifying, which require the reader to 
locate and select items of information in the text 


© Integrating and interpreting, which require the reader 
to process what is read to make internal sense of 
the text, whether by connecting various pieces of 
information or making meaning from something not 
stated 


© = Reflecting and evaluating, which require the reader to 
draw on knowledge, ideas, or attitudes beyond the 
text and to relate them to the text 


Finally, the real-world contexts from which the PISA 2012 
tasks are drawn include work and occupation, personal 
situations (i.e., related to home and family, health, 
shopping, or leisure), community and citizenship, and 
education and training. 


PISA 2012 mathematics literacy assessment 


In PISA 2012, mathematics literacy was defined as 

“an individual’s capacity to formulate, employ, and 
interpret mathematics in a variety of contexts. It includes 
reasoning mathematically and using mathematical 
concepts, procedures, facts and tools to describe, 
explain and predict phenomena. It assists individuals 

to recognise the role that mathematics plays in the 
world and to make the well-founded judgments and 
decisions needed by constructive, engaged and reflective 
citizens” (OECD 2013b, p. 17). This was measured by 
asking respondents to demonstrate their knowledge of 
mathematics content and processes to solve problems in 
a variety of problem scenarios and contexts. 


The mathematical content covered in PISA 2012 includes 
four areas that generally overlap with school-based 
mathematics disciplines: quantity, space and shape, 
change and relationships, and uncertainty and data. 

In terms of the required mathematical processes, the 
PISA 2012 mathematics literacy assessment represents 

a range: formulate situations mathematically; employ 
mathematical concepts, facts, procedures, and reasoning; 
and interpret, apply, and evaluate mathematical 
outcomes. Finally, as with reading literacy, PISA 2012 
mathematics tasks draw from a broad range of real- 
world contexts. For mathematics literacy, these include 
occupational, personal, societal, and scientific. 


Content and skills measured in 
ESO for PISA YAFS at age 19 


ESO was initially developed to provide individual- 

level results that were linked to PIAAC and could be 
obtained at any time, not limited to PIAAC’s decennial 
administration. As such, the content and skills that ESO 
measures are based on the PIAAC assessment framework 
(OECD 2012) and—like PISA 2012—were selected to meet 
the OECD Education Directorate’s overarching goal to 
“identify and develop the knowledge and skills that drive 
better jobs and better lives, generate prosperity and 
promote social inclusion” (OECD n.d.). 


The PIAAC (and thus ESO) framework specifies three 
content domains: literacy, numeracy, and problem 
solving in technology-rich environments. Literacy and 
numeracy were selected as the focus in PISA YAFS 
because these are the two ESO subjects that overlap 
with PISA 2012. ESO is a computer-based, adaptive 
assessment. Participants answer questions in the literacy 
and numeracy content domains. 


ESO literacy assessment 

The ESO literacy assessment is designed to measure 
everyday literacy, which is defined as “understanding, 
evaluating, using and engaging with written texts to 
participate in society, to achieve one’s goals, and to 
develop one’s knowledge and potential” (OECD 2012, 

p. 3). This is accomplished by providing respondents with 
a set of texts about which they must answer questions or 
perform related tasks in various contexts. 


Several types of texts are included in ESO, such as 
continuous text (i.e., prose text) and noncontinuous text 
(i.e., nonprose text that uses explicit typographic features 
to organize information). Additionally, some texts in ESO 


include elements of both continuous and noncontinuous 
text (mixed text), and some present multiple texts. 


The range of cognitive skills required by the questions 
and tasks in the ESO literacy assessment can be 
categorized into three broad strategies that are necessary 
for achieving a full understanding of texts: 


© Accessing and identifying, which require the reader 
to locate items of information in text either through 
locating explicit text or by inferring or having a 
rhetorical understanding of the text 


© Integrating and interpreting, which require the reader 
to understand the relationships between different 
parts of a text, which again may be explicit or may 
require inference 


© Evaluating and reflecting, which require the reader to 
draw on knowledge, ideas, or values external to the 
text 


Finally, the ESO literacy tasks are drawn from a broad 
range of real-world contexts, including work and 
occupation, personal situations (i.e., related to home 
and family, health, shopping, or leisure), community 
and citizenship, and education and training. 


ESO numeracy assessment 


The ESO numeracy assessment is designed to evaluate 
basic mathematical and computational skills that are 
considered fundamental for functioning in everyday 
work and social life. Numeracy is defined as “the ability 
to access, use, interpret and communicate mathematical 
information and ideas, in order to engage in and manage 
the mathematical demands of a range of situations in 
adult life’ (OECD 2012, p. 4). This is accomplished by 
providing respondents with a set of materials or problem 
situations that contain mathematical information, about 
which they must answer questions or perform related 
tasks in various contexts. 


Four key areas of mathematical content, information, 
and ideas are covered in ESO: quantity and number; 
dimension and shape; patterns, relationships, and 
change; and data and chance. Across these key areas, 
mathematical information may be presented in a variety 
of forms, including objects and pictures; numbers and 
mathematical symbols; formulas; diagrams, maps, 
graphs, and tables; texts; and technology-based displays. 


The tasks in the ESO numeracy assessment represent 
a range of numeracy demands requiring mathematical 


processes and responses that can be categorized under 
three broad headings: 


© Identify, locate, or access mathematical information 
that is present in the task or situation and relevant to 
the respondent’s purpose or goal 


© Act upon or use already known mathematical 
procedures or mathematical information such as that 
required in counting, computing, ordering, sorting, 
estimating, measuring, or using or developing 
formulas 


© Interpret, evaluate or analyze, and communicate 
mathematical information 


All ESO numeracy tasks replicate real-world situations 
in which numeracy skills are needed to achieve a goal or 
purpose, and these situations may be related to work, 
everyday life, society or community, or further learning. 


Similarities and differences between 
PISA 2012 and ESO 


As the prior descriptions suggested, despite each 
assessment having its own unique framework, purpose, 
and target population, there are substantial similarities 
in the content and skills measured and the overall 
approach and philosophy between PISA 2012 and ESO/ 
PIAAC (exhibit 1). At a broad level, both measure key 
competencies that are applicable across a wide range of 
culturally and linguistically diverse countries, recognize 
a continuum of competency, and reflect real-world 
demands and contexts.* 


In terms of their frameworks, PISA 2012 and ESO 

also both have the same set of organizing dimensions 
(content, cognitive, and context), and the categories 
within them are similarly named and are, in most cases, 
defined in similar ways. For example, both PISA 2012 and 
ESO define the content dimension of (reading) literacy to 
include continuous, noncontinuous, mixed, and multiple 
texts, and both define the cognitive dimension to include 
accessing and identifying, integrating and interpreting, 
and reflecting and evaluating. This likely owes to their 
largely similar domain definitions, which emphasize 

the deployment of reading skills for pursuit of personal 
goals and development and for participation in society. 
Similarities also exist between PISA 2012 mathematics 
literacy and ESO numeracy, as each identifies four areas 
of content that—despite slightly differing terminology—are 


generally underpinned by knowledge and skills in the 
disciplines of number, geometry, algebra, and probability 
and statistics. 


There are some subtle framework distinctions, however, 
and these are most apparent in mathematics. For 
example, ESO’s definition of numeracy is slightly broader 
than PISA 2012’s definition of mathematics literacy. This 
is because numeracy is rooted in the real-world demands 
in which mathematics plays a role rather than—as 
mathematical literacy is—rooted in mathematical skills 
and how these can be applied in the real world. ESO 

also includes a communication aspect in its mathematics 
cognitive dimension, which could require a respondent 
to describe their mathematical actions or interpretations 
in addition to interpreting or evaluating, as are the limits 
in PISA 2012. Finally, in the context dimension (for both 
literacy and numeracy), ESO’s categories are slightly 
broader than those in PISA 2012 to account for its adult 
(and not exclusively student) population. For example, 
ESO literacy includes an education and training context 
category compared to PISA 2012’s education category in 
reading literacy. 


The most significant potential distinctions in both subjects, 
however, relate less to content and more to the different 
target populations. For example, because PISA 2012 is 
administered to students, it does not have to account 
for as wide a range of skills as are present in the PIAAC/ 
ESO adult population and thus starts at a higher point 
on the performance continuum (Gal and Tout 2014). The 
different populations could also have an impact on how 
the frameworks’ context dimensions are operationalized, 
because the younger PISA target population may not 

be as familiar with the same types of situations as PISA 
YAFS’s 19-year-olds would be, even within similarly 
named context categories. Finally, while neither 
framework is limited to what is typically included in 
school curricula, the tasks in PISA may be somewhat 
more academically oriented than those in ESO. 


In this report, ESO provides the key performance 
outcomes, while PISA 2012 results are examined for 
the possible relationship with the ESO outcomes. For 
simplicity and to avoid confusion with the ESO literacy 
and numeracy assessment, the PISA domains will be 
abbreviated as “PISA 2012 reading” and “PISA 2012 
mathematics” when discussing scores and results. 


4See Borgonovi et al. (2017) for additional detail about the similarities and differences between PIAAC and PISA. The discussion in this section draws 


from this work. See also Gal and Tout (2014). 


Exhibit 1. 


numeracy 


Similarities and differences between PISA 2012 reading and mathematics literacy and ESO literacy and 


ed bY WAM Ar Lae | ed bsY-WAM aac =v-Lellare MELt-var- (ena Lie) PISA 2012 mathematics literacy and 
ESO generally ESO literacy frameworks ESO numeracy frameworks 


Similarities that enable comparisons of the two measures 


Were designed under auspices 
of same organization for same 
overarching goal 


Measure key competencies 
applicable across wide range of 
culturally and linguistically diverse 
countries 


Recognize a continuum of 
competency 


Reflect real-world demands and 
contexts 


Domains are similarly defined 


Share the same set of organizing 
dimensions for frameworks: 


= Content 
= Cognitive 
= Context 


Similarly named and define 
categories within the content and 
cognitive dimensions 


Share the same set of organizing 
dimensions for frameworks: 


= Content 
= Cognitive 
= Context 


Similarly define categories 
(despite variations in naming) 
within the content dimension 


Differences to keep in mind in interpreting results 


PISA student (vs. ESO adult) 
population suggests PISA 
assessment tasks may 


ESO specifies a broader range 
within the context dimension to 
reflect its adult population 


ESO numeracy is slightly more 
broadly defined than PISA 
mathematics literacy 


= Start at higher point on the 
performance continuum 


= Be placed in a more limited 
range of contexts 


= Be somewhat more 
academically oriented 


°® ESO includes a communication 
aspect in its mathematics 
cognitive dimension that PISA 
does not 


° ESO specifies a broader range 
within the context dimension to 
reflect its adult population 


NOTE: PISA 2012 also measured science literacy, problem solving, and financial literacy, and ESO also measured problem solving in technology- 
rich environments. These content areas are not included in the analysis because they were not common across the two assessments and were not 


examined in PISA YAFS. 


SOURCE: OECD 2012, 2013b, 2019a; Borgonovi et al. 2017; Gal and Tout 2014. 


Background information collected by 
PISA 2012 at age 15 and PISA YAFS at 
age 19 


PISA 2012 included background questionnaires: one 
about students and one about schools. The student 
questionnaire collected information on the 15-year-old 
students’ demographic backgrounds, the highest level 
of parental education, and their attitudes and behaviors 
toward school (“feeling happy toward school” and 
frequency of skipping whole days of school). PISA 2012’s 
school questionnaire collected information from school 
administrators on school type (public or private), school 
locale, and students’ receiving free or reduced-price 
lunch (FRPL). These student and school responses were 
used to identify subgroups for analysis. Additionally, this 


report draws on two PISA indices—which were developed 
based on students’ responses to multiple, associated 
questionnaire items—to identify student subgroups for 
analysis. These were the index of economic, social, and 
cultural status and index of openness to problem solving. 
In these cases, students are identified by quarters of the 
index (e.g., students in the bottom quarter are those in 
the lowest 25 percent of values on the index; those in the 
top quarter are those in the highest 25 percent). 


In addition to the ESO content and skills assessments, 
PISA YAFS included a background questionnaire 

that asked the 19-year-old respondents about 

their characteristics, activities, and attitudes. This 
questionnaire used questions from both the ESO 
background questionnaire and a new set of questions 


developed for PISA YAFS that are critical for its 
population.® 


Specific topics included in the questionnaire and 
described in this report are education characteristics, 
employment characteristics, attitudes (i.e., self-efficacy 
and life satisfaction), and vocational interests. The 
questionnaire also asked demographic background 
questions about respondents’ sex, race/ethnicity, nativity, 
and language at home, which are used to analyze PISA 
YAFS results for different subgroups. 


Reporting results 


Average scale scores and proficiency 
levels 


Definitions in PISA 2012 

PISA 2012 results are reported on scales for each 

content domain. The scales for both reading literacy 

and mathematics literacy range from O to 1,000, with a 
standard deviation of 100 points. Using three standard 
deviations up and down from a scale mean of 500, the 
effective scale is 200 to 800 points. The scale averages for 
the countries that participated in 2012 are 493 for reading 
and 490 for mathematics (OECD 2013d). 


PISA 2012 also uses proficiency levels, which divide the 
scales by score-point ranges that are associated with the 
specific sets of knowledge and skills required to complete 
the assessment tasks within the levels (exhibits 2 and 

3). The proficiency levels thus are defined in terms of 
what 15-year-olds at a given level know and can do, and 
the levels show, generally, how proficiency varies along 
the scales. PISA 2012’s proficiency levels are labeled 
numerically (exhibits 2 and 3): from level 1b and 1a at the 
low end to level 6 at the high end of the reading literacy 
scale and from level 1 at the low end to level 6 at the high 
end of the mathematics literacy scale. Additionally, there 
is a “below level 1” category for each domain. 


For the purposes of this report, proficiency levels 

have been combined and thus results are presented 

for below level 2, levels 2 through 4, and level 5 and 
above. This grouping of proficiency-level categories was 
previously used in international reports to designate 
lower performance, middle performance, and higher 
performance, respectively (see, e.g., OECD 2016a). On 
identifying the threshold between lower and middle 


performance, for example, the OECD recently stated that 
students at level 2 are at the “baseline level of proficiency 
that all students should be expected to attain in order 

to take advantage of further learning opportunities and 
participate fully in the social, economic, and civic life of 
modern societies in a globalized world” (OECD 2016a, 

p. 64). 


Definitions in ESO 

ESO results are reported on PIAAC’s scale, which is 
from O to 500 points and represents a continuum of 
proficiency.® There are separate scales for literacy and 
numeracy, each of which has a standard deviation of 50 
points. Using three standard deviations up and down 
from a scale mean of 250, the effective scale is 100 to 
4OO points. The scale averages for the countries that 
participated in PIAAC in 2012 were 272 points in literacy 
and 269 points in numeracy (OECD 2013a). 


ESO also uses PIAAC’s proficiency levels, which are 
defined in terms of what individuals at a given level 
know and can do, and the levels show, generally, how 
proficiency varies along the scales (exhibits 4 and 5). The 
ESO proficiency levels are labeled numerically: levels 1 
through 5 as well as “below level 1” for both literacy and 
numeracy. Respondents at levels 1 to 5 demonstrate not 
only the knowledge and skills associated with a particular 
level but also the proficiencies required at lower levels. 
Respondents who score “below level 1” demonstrate 
proficiency lower than what is needed to accomplish 
successfully at level 1. 


For the purposes of this report, proficiency levels have 
been combined and thus results are presented for below 
level 2, levels 2 and 3, and level 4 and above—which, 
again, were previously used in international reports to 
designate lower performance, middle performance, and 
higher performance, respectively (see, e.g., OECD 2013a, 
2016b). Similar to PISA, the threshold between the low 
and middle groups represents the expected baseline of 
proficiency. 


Uses in this report and notes on comparing 
proficiency levels 

This report uses the ESO average scale scores and the 
percentages of the PISA YAFS population reaching low, 
middle, and high proficiency levels in ESO to provide a 
picture of literacy and numeracy performance at 19 years 
old (in section 3). These results are examined for the 


5 The tailored set of questions was developed based on questions in the PIAAC background questionnaire and is referred to as the Learning 
Experience Questionnaire. It is described in more detail, along with the standard set of ESO questions, in appendix C. 
°For additional detail on how scales were established (including the use of plausible values) and other methodologies, see appendix C. 


overall population as well as for demographic groups by 
sex, race/ethnicity, U.S. nativity, and language spoken at 
home. 


The report uses the PISA 2012 average scale scores 

and the percentages reaching low, middle, and high 
proficiency levels in PISA 2012 to describe how the PISA 
YAFS population’s performance at age 15 relates to ESO 
performance at age 19 (in section 3). In such analyses that 
relate PISA performance at age 15 to ESO performance 

at age 19, results are examined not only by demographic 
variables but also by additional variables that capture 
prior socioeconomic and other conditions at age 15 that 
are relevant for studying relationships over time. 


The percentages of the PISA YAFS population that 
reached the low, middle, and high proficiency levels 
in PISA 2012 are also used to analyze educational, 
employment, and other outcomes at age 19 (section 4). 


The question arises as to the comparability of the 
proficiency levels in the two assessments, and it is 
important to point out that both the individual and 
categorized levels have their own program-specific 
nuances and are not necessarily directly comparable— 
though the categories may broadly indicate similar 
relative performance within the assessments. 


The individual proficiency levels are based on scales 
unique to each specific assessment, and there are a 
different number of levels in each assessment (and 
sometimes in each content domain). The categorized 
levels, too, are based on these unique scales. However, 
because the thresholds for these categorizations were set 
based on the same rationale for both assessments, the 
low, middle, and high levels in each assessment might 

be indicative of broadly similar categories. For example, 
the low-performing category in ESO literacy and PISA 
2012 reading requires only the most basic skills executed 
in familiar texts or contexts with clear instructions, no 
inferring, and minimal distractions present. The high- 
performing category in ESO literacy and PISA 2012 
reading literacy is characterized by items of increased 
complexity and abstractness with increased requirements 
for interpreting, reasoning, or communicating ideas. 
However, again, these similarities are only at the broad 
level, and program-specific nuances should be kept in 
mind and related results interpreted with caution. 


Note on analysis and 
presentation 


All calculations in this report are based on 
unrounded data and thus, in some cases, differences 
cited in the text may differ slightly from calculations 
based on the rounded data that are presented in 
tables and figures. 


All estimates in the report include standard errors, 
which express the sampling and measurement 
variance that could occur if a different sample from 
the same population was used, or if respondents 
answered a different set of tasks in the assessment 
than they did. All differences in percentages 
described in this report were tested using t tests. 
Correlations were tested using the conversion to 
Fisher’s z values and the statistical procedures 

as described in Cohen (1988). For interpreting 
correlation effect sizes, this report uses Cohen’s 
(1988) convention: a correlation coefficient (r) of 
0.1 is small, 0.3 is medium, and 0.5 is large. No 
statistical adjustments to account for multiple 
comparisons were used. 


Differences that are statistically significant at the 
.05 level are discussed using comparative terms 
such as “higher” and “lower.” Differences that are 
not statistically significant are either not discussed 
or referred to as “not statistically significantly 
different.” The failure to find a statistically 
significant difference does not necessarily mean 
that there was not a difference; it could be that 

a real difference could not be detected by the 
significance test because of a small sample size or 
an imprecise measurement. If the statistical test is 
significant, it means that there is no more than a 

5 percent probability that the observed differences 
could be attributed to chance and therefore they 
are likely indicative of true differences in the 
population. However, it is important to remember 
that statistically significant results do not necessarily 
identify those findings that have policy significance 
or practical importance. 


Correlations the correlation statistics for ESO literacy and PISA 2012 
reading literacy and for ESO numeracy and PISA 2012 
mathematics literacy. This statistic varies between 

-1.0 and 1.0, with negative values indicating a negative 
relationship and positive values indicating a positive 
relationship (and values farther from zero indicating a 
stronger relationship than those closer to zero); zero 
indicates no relationship. 


This report also describes the correlation between scale 
scores (expressed in Fisher’s z units) in ESO and PISA 
2012, or the degree to which the two are related and vary 
together. For example, are higher scores in PISA 2012 
associated with higher scores in ESO, with lower scores in 
ESO, or is there no relationship at all? Section 3 presents 


Exhibit 2. Description of PISA 2012 proficiency levels in reading literacy 
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Level 1b 262toless Tasks require the reader to locate a single piece of explicitly stated information in a promi- 
than 335 nent position in a short, syntactically simple text with a familiar context and text type, such 
as a narrative or a simple list. The text typically provides support to the reader, such as rep- 
etition of information, pictures, or familiar symbols. There is minimal completing information. 
In tasks requiring interpretation, the reader may need to make simple connections between 
adjacent pieces of information. 


Level 1a 335toless Some tasks require the reader to locate one or more pieces of information, which may need 
than 407 to be inferred and may need to meet several conditions. Others require recognizing the main 
idea in a text, understanding relationships, or construing meaning within a limited part of the 
text when the information is not prominent and the reader must make low-level inferences. 
Tasks at this level require readers to make a comparison or several connections between 
the text and outside knowledge, by drawing on personal experience and attitudes. 


Level 2 407 toless Tasks require the reader to locate—and, in some cases, recognize the relationship be- 

than 480 tween—several pieces of information that must meet multiple conditions. Interpretative tasks 
at this level require the reader to integrate several parts of a text in order to identify a main 
idea, understand a relationship, or construe the meaning of a word or phrase. They need 
to take into account many features in comparing, contrasting, or categorizing. Often the 
required information is not prominent or there is much competing information, or there are 
other text obstacles, such as ideas that are contrary to expectation or negatively worded. 
Reflective tasks at this level may require connections, comparisons, and explanation, or 
they may require the reader to evaluate a feature of the text. Some reflective tasks require 
the reader to demonstrate a fine understanding of the text in relation to familiar, everyday 
knowledge. Other tasks do not require detailed text comprehension but require the reader to 
draw on less common knowledge. 


Level 3 480 toless Tasks that involve retrieving information require the reader to locate and organize several 
than 553 pieces of embedded information. Some tasks at this level require interpreting the meaning 
of nuances of language in a section of text by taking into account the text as a whole. Other 
interpretative tasks require understanding and applying categories in an unfamiliar context. 
Reflective tasks at this level require the reader to use formal or public knowledge to hypoth- 
esize about or critically evaluate a text. The reader must demonstrate an accurate under- 
standing of long or complex texts whose content may be unfamiliar. 


Level 4 553 toless Tasks that involve retrieving information require the reader to locate and organize several 
than 626 pieces of embedded information. Some tasks at this level require interpreting the meaning 
of nuances of language in a section of text by taking into account the text as a whole. Other 
interpretative tasks require understanding and applying categories in an unfamiliar context. 
Reflective tasks at this level require the reader to use formal or public knowledge to hypoth- 
esize about or critically evaluate a text. The reader must demonstrate an accurate under- 
standing of long or complex texts whose content may be unfamiliar. 


See notes at end of table. 
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Exhibit 2. Description of PISA 2012 proficiency levels in reading literacy—continued 


DY =¥-Yod af eld (ola) 


Level 5 626 toless _ Tasks that involve retrieving information require the reader to locate and organize several 
than 698 pieces of deeply embedded information, inferring which information in the text is relevant. 
Reflective tasks require critical evaluation or hypothesis, drawing on specialized knowledge. 
Both interpretative and reflective tasks require a full and detailed understanding of a text 
whose content or form is unfamiliar. For all aspects of reading, tasks at this level typically 
involve dealing with concepts that are contrary to expectations. 


Level 6 698 or Tasks typically require the reader to make multiple inferences, comparisons, and contrasts 
higher that are both detailed and precise. They require demonstration of a full and detailed under- 

standing of one or more texts and may involve integrating information from more than one 
text. Tasks may require the reader to deal with unfamiliar ideas, in the presence of promi- 
nent competing information, and to generate abstract categories for interpretations. Reflect 
and evaluate tasks may require the reader to hypothesize about or critically evaluate a 
complex text on an unfamiliar topic, taking into account multiple criteria or perspectives and 
applying sophisticated understanding from beyond the text. A salient condition for access 
and retrieve tasks at this level is precision of analysis and fine attention to detail that is 
inconspicuous in the texts. 


' Indicates individual proficiency level definitions and the categories into which the levels have been combined for the purposes of this report. 

NOTE: The scores in the “Score point range” column have been rounded to the nearest whole number. For exact cut scores, see https://nces.ed.gov/ 
surveys/pisa/pisa2012/pisa201 2highlights_5.asp. 

SOURCE: OECD 2013d. 


Exhibit 3. Description of PISA 2012 proficiency levels in mathematics literacy 
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Level 1 358 toless Students can answer questions involving familiar contexts where all relevant information 
than 420 is present and the questions are clearly defined. They are able to identify information and 
to carry out routine procedures according to direct instructions in explicit situations. They 
can perform actions that are almost always obvious and follow immediately from the given 
stimuli. 


Level 2 420 toless Students can interpret and recognize situations in contexts that require no more than direct 
than 482 inference. They can extract relevant information from a single source and make use of a 
single representational mode. Students at this level can employ basic algorithms, formulas, 
procedures, or conventions to solve problems involving whole numbers. They are capable of 
making literal interpretations of the results. 


Level3 482toless Students can execute clearly described procedures, including those that require sequential 
than 545 decisions. Their interpretations are sufficiently sound to be a base for building a simple mod- 
el or for selecting and applying simple problem-solving strategies. Students at this level can 
interpret and use representations based on different information sources and reason directly 
from them. They typically show some ability to handle percentages, fractions, and decimal 
numbers and to work with proportional relationships. Their solutions reflect that they have 
engaged in basic interpretation and reasoning. 


Level 4 545toless Students can work effectively with explicit models for complex concrete situations that may 
than 607 involve constraints or call for making assumptions. They can select and integrate different 
representations, including symbolic, linking them directly to aspects of real-world situations. 
Students at this level can utilize their limited range of skills, and can reason with some 
insight, in straightforward contexts. They can construct and communicate explanations and 
arguments based on their interpretations, arguments, and actions. 


See notes at end of table. 
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Exhibit 3. Description of PISA 2012 proficiency levels in mathematics literacy—continued 


Level 5 607 to less 
than 669 


DY =Y-Yod gf eld (ola) 


Students can develop and work with models for complex situations, identifying constraints 
and specifying assumptions. They can select, compare, and evaluate appropriate prob- 
lem-solving strategies for dealing with complex problems related to these models. Students 
at this level can work strategically using broad, well-developed thinking and reasoning 
skills, appropriate linked representations, symbolic and formal characterizations, and insight 
pertaining to these situations. They begin to reflect on their work and can formulate and 
communicate their interpretations and reasoning. 


Level6 6690r 
higher 


Students can conceptualize, generalize, and utilize information based on their investiga- 
tions and modeling of complex problem situations and can use their knowledge in relatively 
nonstandard contexts. They can link different information sources and representations 

and flexibly translate among them. Students at this level are capable of advanced mathe- 
matical thinking and reasoning. These students can apply this insight and understanding, 
along with a mastery of symbolic and formal mathematical operations and relationships, to 
develop new approaches and strategies for attacking novel situations. Students at this level 
can reflect on their actions and can formulate and precisely communicate their actions and 
reflections regarding their findings, interpretations, arguments, and the appropriateness of 
these to the original situation. 


' Indicates individual proficiency level definitions and the categories into which the levels have been combined for the purposes of this report. 
NOTE: The scores in the “Score point range” column have been rounded to the nearest whole number. For exact cut scores, see https://nces.ed.gov/ 
surveys/pisa/pisa2012/pisa201 2highlights_3.asp. 


SOURCE: OECD 2013d. 


Exhibit 4. Description of ESO proficiency levels in literacy 


Below 176 
level 1 


DY =¥-Yed gfe) (oyal 


The tasks at this level require the respondent to read brief texts on familiar topics to locate 
a single piece of specific information. There is seldom any competing information in the text, 
and the requested information is identical in form to information in the question or directive. 
The respondent may be required to locate information in short continuous texts. However, 

in this case, the information can be located as if the text were noncontinuous in format. Only 
basic vocabulary knowledge is required, and the reader is not required to understand the 
structure of sentences or paragraphs or make use of other text features. Tasks below Level 
1 do not make use of any features specific to digital texts. 


Level 1 176 to less 
than 226 


Most of the tasks at this level require the respondent to read relatively short digital or print 
continuous, noncontinuous, or mixed texts to locate a single piece of information that is iden- 
tical to or synonymous with the information given in the question or directive. Some tasks, 
such as those involving noncontinuous texts, may require the respondent to enter personal 
information into a document. Little, if any, competing information is present. Some tasks may 
require simple cycling through more than one piece of information. Knowledge and skill in 
recognizing basic vocabulary determining the meaning of sentences, and reading paragraphs 
of text, is expected. 


Level 2 226 to less 
than 276 


At this level, the medium of texts may be digital or printed, and texts may comprise contin- 
uous, noncontinuous, or mixed types. Tasks at this level require the respondent to make 
matches between the text and information and may require paraphrasing or low-level 
inferences. Some competing pieces of information may be present. Some tasks require the 
respondent to cycle through or integrate two or more pieces of information based on criteria, 
compare and contrast or reason about information requested in the question, or navigate 
within digital texts to access and identify information from various parts of a document. 


See notes at end of table. 
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Exhibit 4. Description of ESO proficiency levels in literacy—continued 


Middle Level 3 276 to less 
than 326 


Description 


Texts at this level are often dense or lengthy, and include continuous, noncontinuous, mixed, 
or multiple pages of text. Understanding text and rhetorical structures becomes more central 
to successfully completing tasks, especially navigating complex digital texts. Tasks require 
the respondent to identify, interpret, or evaluate one or more pieces of information and 

often require varying levels of inference. Many tasks require the respondent to construct 
meaning across larger chunks of text or perform multistep operations in order to identify and 
formulate responses. Often tasks also demand that the respondent disregard irrelevant or 
inappropriate content to answer accurately. Competing information is often present, but it is 
not more prominent than the correct information. 


Level 4 326 to less 
than 376 


Tasks at this level often require the respondent to perform multistep operations to integrate, 
interpret, or synthesize information from complex or lengthy continuous, noncontinuous, 
mixed, or multiple type texts. Complex inferences and application of background knowledge 
may be needed to perform the task successfully. Many tasks require identifying and under- 
standing one or more specific, noncentral ideas in the text in order to interpret or evaluate 
subtle evidence-claim or persuasive discourse relationships. Conditional information is 
frequently present in tasks at this level and must be taken into consideration by the respon- 
dent. Competing information is present and sometimes seemingly as prominent as correct 
information. 


Level5 376or 
higher 


At this level, tasks may require the respondent to search for and integrate information 
across multiple, dense texts; construct syntheses of similar and contrasting ideas or points 
of view; or evaluate evidence-based arguments. Application and evaluation of logical and 
conceptual models of ideas may be required to accomplish tasks. Evaluating reliability of 
evidentiary sources and selecting key information is frequently a requirement. Tasks often 
require the respondent to be aware of subtle, rhetorical cues and make high-level inferences 
or use specialized background knowledge. 


' Indicates individual proficiency level definitions and the categories into which the levels have been combined for the purposes of this report. 


SOURCE: OECD 2013a. 


Exhibit 5. Description of ESO proficiency levels in numeracy 


Below Below 176 
level 1 
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Tasks at this level require the respondent to carry out simple processes, such as counting, 
sorting, performing basic arithmetic operations with whole numbers or money, or recogniz- 
ing common spatial representations in concrete, familiar contexts where the mathematical 
content is explicit with little or no text or distractors. 


Level 1 176 to less 
than 226 


Tasks at this level require the respondent to carry out basic mathematical processes in com- 
mon, concrete contexts where the mathematical content is explicit with little text and minimal 
distractors. Tasks usually require one-step or simple processes involving counting, sorting, 
performing basic arithmetic operations, understanding simple percents (such as 50 percent), 
and locating and identifying elements of simple or common graphical or spatial representa- 
tions. 


Level 2 226 to less 
than 276 


Tasks at this level require the respondent to identify and act on mathematical information 
and ideas embedded in a range of common contexts where the mathematical context is 
fairly explicit or visual with relatively few distractors. Tasks tend to require the application 
of two or more steps or processes involving calculations with whole numbers and common 
decimals, percents, and fractions; simple measurement and spatial representation; estima- 
tion; and interpretation of relatively simple data and statistics in texts, tables, and graphs. 


See notes at end of table. 
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Exhibit 5. Description of ESO proficiency levels in numeracy—continued 


Level! 


Description 


Middle Level 3 276 to Tasks at this level require the respondent to understand mathematical information that may 
less than _ be less explicit, embedded in contexts that are not always familiar, and represented in more 
326 complex ways. Tasks require several steps and may involve the choice of problem-solving 
strategies and relevant processes. Tasks tend to require the application of number sense and 
spatial sense; recognizing and working with mathematical relationships, patterns, and propor- 
tions expressed in verbal or numerical form; and interpretation and basic analysis of data and 
statistics in texts, tables, and graphs. 

Level 4 326 Tasks at this level require the respondent to understand a broad range of mathematical 
points to information that may be complex, abstract, or embedded in unfamiliar contexts. These tasks 
less than _ involve undertaking multiple steps and choosing relevant problem-solving strategies and 
376 processes. Tasks tend to require analysis and more complex reasoning about quantities and 

data; statistics and chance; spatial relationships; and change, proportions, and formulas. 
Tasks at this level may also require understanding arguments or communicating well-rea- 
soned explanations for answers or choices. 

Level 5 376 or Tasks at this level require the respondent to understand complex representations and abstract 
higher and formal mathematical and statistical ideas, possibly embedded in complex texts. The 


respondent may have to integrate multiple types of mathematical information where consider- 
able translation or interpretation is required; draw inferences; develop or work with mathemati- 


cal arguments or models; and justify, evaluate, and critically reflect on solutions or choices. 


' Indicates individual proficiency level definitions and the categories into which the levels have been combined for the purposes of this report. 


SOURCE: OECD 2013a. 


PISA YAFS population 


Sampling procedures 


The PISA YAFS starting sample was the pool of 15-year-old 
students who took the PISA 2012 mathematics, reading, 
and science assessments and completed a Student 
Information Form (SIF) with their contact information.’ 
Nearly 5,000 students took the PISA 2012 mathematics, 
reading, and science assessments, and 93 percent of 
them completed the SIF, resulting in a PISA YAFS starting 
sample of 4,612 students (exhibit 6). 


These students were contacted periodically over the next 
4 years through an extensive tracing and recruitment 
effort to maximize participation in PISA YAFS, which 

was voluntary. The PISA YAFS final sample included 
2,318 students, which represented 50 percent of the 
starting sample. The primary reason for nonparticipation 
was a lack or loss of contact with the students after 

they completed the SIF, sometime during the tracing 

and recruitment effort. This was the case for nearly 

all nonparticipants, although a small number (1) were 
excluded (due to lack of access to a computer, serving in 
the military abroad, or serving on a church mission) or 
(2) explicitly refused to participate. 


Because of the level of nonparticipation in the final 
sample, nonresponse bias analyses were undertaken 

to identify any systematic differences between 
participants and nonparticipants. The study then made 
weighting adjustments to reduce any bias detected. 

(The biases detected and the adjustments made are 
described in appendix C.) As a result, the participants 
and nonparticipants (or starting and final samples) are 
considered comparable, and the results reported are 
weighted estimates for the PISA YAFS population. It should 
also be noted that although ESO literacy and numeracy 
assessments were administered together, a small number 
of PISA YAFS participants completed only literacy or only 
numeracy items, so the characteristics (and performance) 
of the student populations are estimated separately for 
each subject and are described as the “YAFS literacy” or 
“YAFS numeracy” population in this report. 


The PISA 2012 sample—and thus the pool of students 
on which the PISA YAFS samples was based-reflected 
a two-stage design. First, schools were sampled from 
the universe of U.S. public and private schools that 
contained at least one 7th- through 12th-grade class; 
second, 50 eligible students were sampled within those 
schools. Eligible students were between 15 years and 


7 This excluded PISA 2012 students who took the financial literacy assessment. These students instead participated in the field test of the PISA 
YAFS instruments (ESO assessment, ESO background questionnaire, and supplemental background questions added from the PIAAC background 


questionnaire). 


Exhibit 6. PISA YAFS sample selection process 


PISA 2012 students 
(excluding financial literacy) 


N = 4,978 


Se 


Returned SIF/eligible for 
PISA YAFS in 2016 


N = 4,612 (93% of PISA 2012 students) 


Did not return SIF/not eligible for 
PISA YAFS in 2016 


N = 366 (7% of PISA 2012 students) 


Participated in PISA YAFS in 2016 


N = 2,318 (50% of eligible) 


Did not participate in PISA YAFS in 2016 
N = 2,294 (50% of eligible) 


NOTE: SIF = Student Information Form. Every PISA 2012 student who returned a SIF was contacted. Nonparticipants 
were mainly those who did not respond to any contact attempts or with whom the study lost contact during the tracing and 
recruitment effort. Nonparticipants also include small percentages who were excluded or refused to participate. See appendix 


table C-1 for additional detail. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student 
Assessment (PISA), 2012; and Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


3 months and 16 years and 2 months at the beginning 

of the assessment period (fall/winter 2012) and had 

basic English language literacy and the ability to access 
the assessment with the allowed accommodations. 
Schools were sampled proportionately to their estimated 
enrollment of 15-year-olds, and students were sampled 
randomly from the eligible population in each school.® 
The PISA 2012 sample is representative of the U.S. 
population of 15-year-olds in 2012.° 


Because the original PISA 2012 sample was nationally 
representative of the 15-year-old population, and because 
of the nonresponse bias adjustments in the current study, 
the PISA YAFS population is nationally representative of 
individuals who were approximately 19 years old in 2016 
and who were 15-year-old students in 2012. Throughout 
the report, the PISA YAFS population is generally referred 
to as such or as “U.S. 19-year-olds,” which describes 
people who are about 19 years old and who had attended 
school at age 15. 


Characteristics of the PISA YAFS 
population 

Table 1 describes the performance and demographic 
characteristics of the PISA YAFS population. Of note, the 
PISA YAFS literacy population had an average PISA 2012 
reading score of 507 points and the numeracy population 
had an average PISA 2012 mathematics score of 491 points 
when they were 15. Neither of the scores were statistically 
significantly different from the respective U.S. averages 

in PISA 2012 (for U.S. averages not shown, see Kelly et 

al. 2013). Additionally, in terms of nativity, 92 percent 

of the PISA YAFS population were born in the United 
States, including those whose parents were also born 

in the country and those whose parents were not (i.e., 
“first generation”). About 86 percent of the PISA YAFS 
population spoke English as their primary language at 
home, with the remaining population speaking either 
Spanish or another language. 


8 Each student did not take all but rather a combination of the subjects included in PISA 2012 (reading, mathematics, etc.), the information on which 


was used to impute a score for the population. 


° These students were mostly in grades 9, 10, and 11, with grade 10 being the modal grade (71 percent). 


Table 1. Performance and demographic characteristics of the PISA and PISA YAFS populations: 2012 and 2016 


PISA YAFS 2016' 
PISA 2012 ESO Literacy ESO Numeracy 


Demographic characteristics [Standard errors appear in parentheses] 
Average PISA 2012 reading score 498 (3.74) 507 (4.86) — (t) 
PISA 2012 reading proficiency (percent) 
Low (Below level 2) 16.6 (1.26) 13.7 (1.99) — (t) 
Middle (Levels 2-4) 75.5 (1.05) 77.5 (1.82) — (t) 
High (Level 5 and above) 7.9 (0.67) 8.9 (0.83) — (t) 
Average PISA 2012 mathematics score 481 (3.60) — (t) 491 (4.55) 
PISA 2012 mathematics proficiency (percent) 
Low (Below level 2) 25.8 (1.39) — (t) 22.4 (2.13) 
Middle (Levels 2—4) 65.4 (1.13) — (t) 67.6 (1.92) 
High (Level 5 and above) 8.8 (0.78) — (t) 10.1 (1.02) 


Demographic characteristics (percent) 


Gender 
Male 51.0 (0.72) 51.1 (1.56) 51.4 (1.58) 
Female 49.0 (0.72) 48.9 (1.56) 48.6 (1.58) 
Race/ethnicity 
White 51.1 (2.37) 52.5 (2.53) 52.7 (2.58) 
Black or African American 12.7 (1.33) 12.3 (1.72) 12.5 (1.71) 
Hispanic or Latino 24.8 (1.81) 23.0 (1.75) 22.9 (1.72) 
Asian 4.8 (0.84) 5.8 (1.28) 5.8 (1.29) 
Other 6.6 (0.81) 6.4 (1.07) 6.1. (1.05) 
Born in the United States 
Native 77.4 (1.96) 77.1 (2.36) 77.1 (2.30) 
First-generation native 14.6 (1.35) 15.2 (1.64) 15.1 (1.61) 
Nonnative 8.0 (0.81) 7.7 (1.06) 7.8 (1.02) 
Language at home 
English 85.6 (1.30) 86.4 (1.35) 86.3 (1.30) 
Spanish 10.9 (1.21) 9.4 (1.04) 9.4 (1.01) 
Other languages 3.4 (0.44) 4.2 (0.73) 4.2 (0.73) 
— Not available. 


t Not applicable. 

' Although the ESO literacy and numeracy assessments are administered together, a small number of PISA YAFS participants completed only literacy or 
only numeracy items, and, thus, the scales and populations are estimated separately. 

NOTE: ESO stands for Education and Skills Online. Estimates for the PISA YAFS population (19-year-olds) are for individuals who were 15-year-old 
students in the fall of 2012. As part of PISA YAFS, participants completed the ESO assessments in 2016. Race categories exclude persons of Hispanic 
ethnicity. “Other” includes those who identified themselves as “Two or more races,” “American Indian/Alaska Native,” and “Native Hawaiian/Other Pacific 
Islander.” There are no statistically significant differences between the percentages in the PISA YAFS 2016 and PISA 2012 populations by demographic 
characteristics. The percentages of the PISA YAFS 2016 population who had low proficiency (below level 2) or middle proficiency (levels 2—4) in PISA 
2012 are significantly different than the percentages of the PISA 2012 population in the respective levels (p < .05). See exhibits 2 and 3 for descriptions 
of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Comparisons of characteristics between PISA 
YAFS students and the PISA 2012 population 


Statistically, the PISA YAFS population was similar to the 
original PISA 2012 population. There were no statistically 
significant differences between the two in average PISA 
2012 performance in reading or mathematics or in the 
distribution across 2012 proficiency levels (table 1). 
Additionally, there were no statistically significant 
differences between the two populations by any of the 
demographic characteristics examined. This reinforces 
that the PISA YAFS population is a valid representation of 
the PISA 2012 population. 


Throughout the report, the PISA YAFS population is 
referred to as the “population overall” when being 
compared with student subgroups of the PISA YAFS 
population. 


Data limitations 


In considering the results of PISA YAFS, it is important to 
understand a few caveats. 


First, each assessment used in PISA YAFS was developed 
independently and for distinct purposes. Thus, the 
results for each assessment are placed on its own scale 
in each subject. Therefore, while readers can compare 
general trends in scale scores, they should not compare 
exact scale scores and average scale scores between 
subjects within assessments, nor within subjects across 
the two assessments. This means that while proficiency 
levels of equivalent labeling (e.g., level 2) are similar 
enough for general comparisons, they should not be 
considered fully equivalent. These broad relationships 
explain why grouping of proficiency levels into high, 
middle, and low categories indicative of broadly similar 
categories because the thresholds were set based on 
similar rationales are still appropriate for comparison. 
However, these similarities are only at the broad level, 
and program-specific nuances should be kept in mind 
and related results interpreted with caution. 
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Second, while both assessments were voluntary, the 
nature of the sample selection processes led to different 
sources of nonparticipation. Because PISA 2012 first 
drew a school sample and then sampled students within 
schools, nonparticipation tended to be at the school 
level. In PISA YAFS, all students who took the PISA 2012 
reading, mathematics, and science assessments and 
completed an SIF were eligible to take ESO; therefore, all 
nonparticipation was at the individual level. While there 
were differences between the starting and final PISA 
YAFS samples related to background characteristics, the 
study’s sampling adjustments mitigated nearly all of them 
(differences of less than 1 percentage point remained for 
preschool attendance and educational expectations; see 
appendix C). 


Finally, there were differences in the mode of assessment. 
PISA 2012’s assessments were administered via paper 

and pencil in the classroom, whereas ESO was computer- 
based and self-administered by participants in their 
home or a place with access to a computer. While the 
equivalence of scoring the assessment items between 
paper-and-pencil and computer formats have been 
examined and reported for ESO/PIAAC and PISA, no 
examination of the classroom versus home settings have 
taken place (OECD 2010b, 2019b). 


For additional detail on how scales were established 
(including the use of plausible values) and other 
methodologies, see appendix C. 


3. Performance in ESO at 
Age 19 and How It Relates to 
PISA Performance at Age 15 


This section examines how U.S. 19-year-olds performed in 
the ESO literacy and numeracy assessments administered 
in 2016 in PISA YAFS—in terms of average score and 
proficiency level—and how proficiency relates to their 
proficiency in reading and mathematics 4 years earlier in 
PISA 2012. 


It opens with a description of their average performance 
in ESO and the percentages at different proficiency 
levels. It then examines the distributions across ESO 
proficiency levels based on PISA 2012 proficiency— 
asking how students of different PISA 2012 proficiency 
levels later performed as 19-year-olds in ESO. In these 
analyses, the report uses the groupings of proficiency 
levels that designate low, middle, and high performance. 
As described earlier (on p. 9), both the individual and 
grouped proficiency levels have their own program- 
specific definitions and are not necessarily directly 
comparable; however, the grouped proficiency levels 
might be indicative of similar relative performance in 
each assessment. This section concludes with correlation 
analyses asking how closely related individuals’ scores on 
the two assessments were. 


The estimates and standard errors for the statistics 
presented in this section can be found in appendix A. 


Average scores and 
proficiency in ESO at age 19 


U.S. 19-year-olds had an average score of 266 points in 
ESO literacy, with scores ranging from a low of 80 points 
to a high of 419 points (table A-1). Overall, 19 percent of 
U.S. 19-year-olds were low performing (i.e., below level 2), 
72 percent were middle performing (i.e., at level 2 or 3), 
and 10 percent were high performing (i.e., at level 4 or 


above) in ESO literacy (table A-2). Looking at subgroups 
of 19-year-olds, there were no differences in the 
percentages at these proficiency levels by sex, race/ 
ethnicity, language spoken at home, or whether they 
were born in the United States. 


In ESO numeracy, U.S. 19-year-olds had an average score 
of 260 points, with scores ranging from a low of 91 points 
to a high of 419 points (table A-3). Overall, 25 percent 

of U.S. 19-year-olds were low-performing (i.e., below 
level 2), 68 percent were middle-performing (i.e., at 

level 2 or 3), and 7 percent were high-performing (i.e., 

at level 4 or above) in ESO numeracy (table A-4). As in 
literacy, there were no differences in the percentages 

at these proficiency levels by sex, language spoken at 
home, or whether they were born in the United States, 
but there was a difference by race/ethnicity. In numeracy, 
proportionately more Black 19-year-olds were low 
performing (45 percent) than the population overall 

(25 percent). 


Relating proficiency in ESO 
at age 19 to proficiency in 
PISA 2012 at age 15 


As it was at age 19 (tables A-2 and A-4), performance at 
age 15 (in 2012) was predominantly in the middle level of 
proficiency. For example, 77 percent of U.S. 19-year-olds 
performed at the middle level of proficiency in PISA 2012 
reading at age 15 and 68 percent performed at the middle 
level of proficiency in PISA 2012 mathematics at age 15 
(table 1). Moreover, looking across the time points, most 
U.S. 19-year-olds who were middle performers at age 15 
remained middle performers at age 19.'° 


© As a reminder, the grouped proficiency levels (low, middle, and high) were based on models used in international reporting, which set thresholds 
according to demonstrated skills rather than to the performance distribution. Although for most international participants the middle group was the 
largest group, the percentages in this group ranged widely (e.g., from 33 to 82 percent of 16- to 34-year-olds in the countries participating in PIAAC 
literacy between 2012 and 2017, and from 40 to 86 percent in the countries in PISA 2012). 


ESO literacy at age 19 and PISA 2012 
reading at age 15 


Seventy-seven percent of the PISA YAFS population had 
performed at the middle proficiency levels (levels 2 to 4) 
of PISA 2012 reading at age 15 (table 1 and figure 1). Of this 
group, about three-quarters (78 percent) also performed 
at the middle level (level 2 or 3) of ESO literacy at age 

19 (figure 1). Seven percent of middle performers in 

PISA 2012 reading performed at the high proficiency 
level (level 4 and above) in ESO literacy, and 15 percent 
demonstrated performance at the low level (below level 
2) at age 19. This is represented by the majority of the red 
bar remaining in the middle proficiency level between 
PISA 2012 and ESO in 2016, and relatively smaller 
segments moving to other proficiency levels (figure 1). 


The pattern for high and low performers remaining in 
the same-labeled proficiency level categories 4 years later 
was not as stark. For example, 9 percent of the PISA YAFS 


Figure 1. 


population had performed at the high proficiency level 
(level 5 and above) of PISA 2012 reading at age 15. Of this 
group, 41 percent also performed at the high level (level 
4 and above) in the ESO literacy assessment at age 19. 
However, 57 percent of the high performers in PISA 2012 
reading performed at the middle level of proficiency 
(levels 2 and 3) in ESO literacy. 


Additionally, 14 percent of the PISA YAFS population had 
performed at the low proficiency level (below level 2) on 
PISA 2012 reading at age 15. Of this group, 51 percent also 
performed at the low proficiency level (below level 2) 

in ESO literacy at age 19. Forty-nine percent of the low 
performers in PISA 2012 reading performed at the middle 
proficiency (levels 2 and 3) in ESO literacy. In other 
words, about half of the students who demonstrated low 
proficiency in reading as 15-year-olds were no longer 

in the low proficiency level (below level 2) in the ESO 
literacy assessment as 19-year-olds. 


Percentage distribution of U.S. 19-year-olds in PISA 2012 reading proficiency levels at age 15 and in ESO 


literacy proficiency levels at age 19: 2012 and 2016 


PISA 2012 (Age 15) 


High Level 5 and above 
(9%) 


. Levels 2—4 
Middle (77%) 
Low Below level 2 


(14%) 


ESO (Age 19) 


Level 4 and above 
(10%) 


Levels 2-3 
(72%) 


Below level 2 
(19%) 


HOW TO READ THE FIGURE: At age 15, nine percent of U.S. 19-year-olds performed at the high level of proficiency in PISA 2012 (i.e., level 5 and 
above). Of this group, 41 percent were in the high level of proficiency in ESO (level 4 and above) at age 19. Altogether, 10 percent of U.S. 19-year-olds 
were at the high level of proficiency (level 4 and above) in ESO. 


$ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. Percentages of U.S. 19-year-olds by their PISA and ESO proficiency levels appear in parentheses below the 
category labels. See tables 1, A-2, and A-5 for standard errors of the estimates. See exhibits 2 and 4 for descriptions of proficiency levels. Detail may 
not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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When examined separately for different subgroups, 

the pattern of a predominant and stable middle group 
held true, largely regardless of 19-year-olds’ background 
characteristics. Compared to the overall population 
(table A-5), there were no statistically significant 
differences between males and females (table A-6), by 
socioeconomic background—as measured by PISA’s 
index of economic, social, and cultural status (ESCS) 
(table A-7)—or by race/ethnicity (table A-9). 


The only observed difference was by the free or reduced- 
price lunch (FRPL) status of U.S. 19-year-olds’ schools 
when they were 15, which is an indicator of economic 
resources in schools’ student populations. Among the 
middle performers in PISA 2012 reading who were in the 
most economically challenged circumstances—schools 
with 75 percent or more of students receiving FRPL— 
proportionately more were at the low proficiency level 
relative to the population overall (28 vs. 15 percent, 
respectively) (tables A-5 and A-8). 


ESO numeracy at age 19 and PISA 2012 
mathematics at age 15 


Patterns in the relationship between mathematics and 
numeracy were similar to those between reading and 
literacy, with proficiency most closely related among the 
middle performers. 


Sixty-eight percent of the PISA YAFS population had 
performed at the middle proficiency level (levels 2 to 

4) of PISA 2012 mathematics at age 15 (table 1, figure 2). 
Of this group, 79 percent also performed at the middle 
level (levels 2 to 3) of ESO numeracy at age 19 (figure 2). 

A small percentage (4 percent) of middle performers on 
PISA 2012 mathematics performed at the high proficiency 
level (level 4 and above) in ESO numeracy, and 17 percent 
demonstrated low performance (below level 2). 
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Ten percent of the PISA YAFS population had performed 
at the high proficiency level (level 5 and above) of PISA 
2012 mathematics at age 15. Of this group, 46 percent also 
performed at the high level (level 4 and above) of the ESO 
numeracy assessment at age 19. However, 54 percent of 
the high performers in PISA 2012 mathematics performed 
at the middle level (levels 2 and 3) of proficiency in ESO 
numeracy. 


Twenty-two percent of the PISA YAFS population had 
performed at the low proficiency level (below level 2) 

on PISA 2012 mathematics at age 15. Of this group, 60 
percent also performed at the low level (below level 2) in 
ESO numeracy at age 19. However, 40 percent performed 
at the middle level (level 2 or 3) of ESO proficiency at age 
19. In other words, about two-fifths of low performers 

on PISA 2012 mathematics at age 15 were able to 
demonstrate higher proficiency relative to the ESO 
numeracy assessment at age 19. 


As with reading and literacy, when comparing the 
patterns of distribution for PISA 2012 mathematics and 
ESO numeracy at age 19 between the overall population 
(table A-10) and various subgroups, no differences were 
observed by sex (table A-11), the ESCS index (table A-12), 
or race/ethnicity (table A-14). 


Again, observed differences were related to the FRPL 
status of 19-year-olds’ schools at age 15. Among the 
middle performers in PISA 2012 mathematics who were 
in the most economically challenged circumstances— 
schools with 75 percent or more of their students 
receiving FRPL—proportionately fewer performed at the 
middle proficiency level (levels 2 and 3) in ESO numeracy 
relative to the population overall (62 vs. 79 percent, 
respectively). Proportionately more performed at the 
low level of proficiency (below level 2) relative to the 
population overall (36 vs. 17 percent, respectively) 
(tables A-10 and A-13). 


Figure 2._ Percentage distribution of U.S. 19-year-olds in PISA 2012 mathematics proficiency levels at age 15 and in 


ESO numeracy proficiency levels at age 19: 2012 and 2016 


PISA 2012 (Age 15) ESO (Age 19) 


Level 4 and above 


High Level 5 and above | 
(7%) 


(10%) 


; Levels 2-4 Levels 2-3 
Middle (68%) (68%) 
Low Below level 2 Below level 2 
(22%) (25%) 


HOW TO READ THE FIGURE: At age 15, ten percent of U.S. 19-year-olds performed at the high level of proficiency in PISA 2012 (i.e., level 5 and 
above). Of this group, 46 percent were in the high level of proficiency in ESO (level 4 and above) at age 19. Altogether, 7 percent of U.S. 19-year-olds 
were at the high level of proficiency (level 4 and above) in ESO. 


+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. Percentages of U.S. 19-year-olds by their PISA and ESO proficiency levels appear in parentheses below the 
category labels. See tables 1, A-4, and A-10 for standard errors of the estimates. See exhibits 3 and 5 for descriptions of proficiency levels. Detail may 
not sum to totals due to rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Overall, the correlation between ESO literacy at age 19 
and PISA 2012 reading at age 15 is 0.59, which indicates 
a strong positive relationship (figure 3 and table A-15)." 
That is, U.S. 19-year-olds with higher literacy scores in 
ESO tended to have had higher reading scores in 2012 


Correlations between ESO 
scores at age 19 and PISA 
2012 scores at age 15 


This section describes the correlation between scale 
scores in ESO and PISA 2012 to determine what, if 

any, relationship there is between scores on the two 
assessments over time. Correlation describes the degree 
to which the two are related and vary together. 


at age 15. The correlation between ESO and PISA 2012 
mathematics and ESO numeracy is 0.69, which indicates 
an even stronger positive relationship (figure 4 and 
table A-16). 


"For interpreting effect sizes, this report uses Cohen’s (1988) convention, for lack of other relevant benchmark metrics. Cohen has shown that a 


correlation coefficient (r) of 0.1 is small, 0.3 is medium, and 0.5 is large. 


Figure 3. Correlation of PISA 2012 reading score and ESO literacy score for U.S. 19-year-olds: 2012 and 2016 


ESO literacy score 


500 


450 


400 


350 


300 


250 


200 


100 200 300 400 500 600 700 800 900 1,000 
PISA 2012 reading score 


NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. Scatterplot is based on the first plausible value for PISA 2012 reading and the first plausible value for ESO 
literacy. For more information on the plausible values, refer to appendix C. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Figure 4. Correlation of PISA 2012 mathematics score and ESO numeracy score for U.S. 19-year-olds: 2012 and 2016 
ESO numeracy score 
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NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. Scatterplot is based on the first plausible value for PISA mathematics and the first plausible value for ESO 
numeracy. For more information on the plausible values, refer to appendix C. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Demographics 

For most demographic subgroups, the strength of the 
correlation between ESO at age 19 and PISA 2012 at age 
15 was not statistically significantly different from that 
of the overall PISA YAFS population. In fact, there were 
no statistically significant differences in the correlation 
statistics for U.S. 19-year-olds by sex, language spoken at 
home, or whether they were born in the United States 
for either reading/literacy or mathematics/numeracy 
(tables A-15 and A-16). Additionally, there were no 
statistically significant differences in the correlation 
statistics for U.S. 19-year-olds by race/ethnicity for 
reading/literacy. 


The only differences from the overall PISA YAFS 
population were in the correlation between mathematics 
and numeracy for Asian and for White 19-year-olds 

(table A-16), who demonstrated opposite effects from 
each other. For Asian 19-year-olds, the correlation 
between ESO numeracy and PISA 2012 mathematics 
(0.79) was stronger than for the population overall (0.69). 
For White 19-year-olds, the correlation between ESO 
numeracy and PISA 2012 mathematics (0.65) was weaker 
than for the population overall (0.69). 


Socioeconomic background 


Two socioeconomic background characteristics were 
related to the strength of the correlations between ESO at 
age 19 and PISA at age 15. The most consistent differences 
were seen related to schools’ FRPL status (tables A-17 and 
A-18). For U.S. 19-year-olds who had attended schools 
with either less than 25 percent or with 75 percent or 
more of students receiving FRPL at age 15, correlations 
were weaker than for the population overall in both 
subjects. For these subgroups, the correlation between 
ESO literacy at age 19 and PISA 2012 reading at age 15 was 
0.50 for students in the least economically challenged 
schools and 0.48 for those in the most economically 
challenged schools, compared to 0.59 for the population 
overall. The correlation between ESO numeracy at age 

19 and PISA 2012 mathematics at age 15 was 0.63 for 
students in the least economically challenged schools 
and 0.54 for those in the most economically challenged 
schools, compared to 0.69 for the population overall. 
That is, PISA 2012 performance at age 15 was less strongly 
related to ESO performance at age 19 for these groups of 
19-year-olds than for the PISA YAFS population overall. 


The other difference in correlation statistics from the 
overall population was related to school locale and 
only for the mathematics/numeracy correlation. The 
correlation between ESO numeracy at age 19 and PISA 
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2012 mathematics at age 15 was stronger for those 

who had attended schools in cities (100,000 to about 
1,000,000 people) (0.76) than for the population overall 
(0.69) (table A-18). In contrast, the correlation was weaker 
for those whose schools were in villages (fewer than 
3,000 people) (0.54) than for the population overall. 


Behavioral and affective characteristics 


As with other characteristics, there were few differences 
in the strength of the correlations between ESO and PISA 
2012 by behavioral and affective characteristics. There 
were no statistically significant differences in correlation 
statistics for U.S. 19-year-olds by their frequency of 
skipping school or their happiness (or lack) at school for 
either the reading/literacy or the mathematics/numeracy 
correlations (tables A-19 and A-20). Additionally, there 
were no measurable differences in correlation statistics 
for reading/literacy related to 19-year-olds’ openness to 
problem solving, which is described by an index created 
from student responses to questions asking about the 
extent to which they feel they resemble someone who 
can handle a lot of information, is quick to understand 
things, seeks explanations for things, can easily link facts 
together, and likes to solve complex problems. 


The only difference from the overall population was for 
the numeracy/mathematics correlation for students in the 
bottom and second-to-bottom quarters of the index on 
openness to problem solving (table A-20). U.S. 19-year-olds 
who had been in these quarters in PISA—who represent 
those expressing the least openness to problem solving— 
had weaker correlations between ESO numeracy at age 19 
and their PISA mathematics scores at age 15 (0.60 for both 
quarters) than the population overall (0.69). 


Looking across performance 


outcomes 


Looking across different subgroups of the PISA YAFS 
population, there are only a few subgroups for whom the 
relationship between proficiency at age 15 and 19 differed 
from the overall pattern or who exhibited different 
strengths of correlations between scores on the two 
assessments. Exhibit 7 summarizes the movements of the 
middle performers in PISA 2012 (i.e., levels 2-4) into the 
low and middle levels of ESO proficiency, identifying the 
one subgroup for whom patterns differed from the PISA 
YAFS population overall. Exhibit 8 presents information 
on the subgroups that showed a stronger or weaker 
correlation between PISA 2012 reading and ESO literacy 
scores and PISA 2012 mathematics and ESO numeracy 
scores, respectively. 


Exhibit 7. Percentage of 19-year-old middle performers in PISA 2012, by ESO proficiency level and subject: 2016 


Mo) a1 OM ol coy ile (Valea Middle ESO proficiency 


in 2016 in 2016 
ESO =SS10) =S<10) ESO 
literacy numeracy literacy numeracy 
U.S. PISA YAFS population overall 15 17 78 79 
From schools with 75 percent or more of students 28 A 36 A 68 62 V 
receiving free or reduced-price lunch 


A Percentage is significantly higher than the PISA YAFS population overall. 
V Percentage is significantly lower than the PISA YAFS population overall. 


NOTE: ESO = Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in fall 2012 and who participated 
in PISA YAFS in 2016. Middle proficiency in PISA 2012 indicates levels 2 to 4. See exhibits 2 through 5 for descriptions of proficiency levels. Data for 
students eligible for free or reduced-price lunch were available for public schools only. For data for other subgroups, see tables A-6 to A-9 for literacy/ 
reading and A-11 to A-14 for numeracy/mathematics. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment Young Adult Follow-up 
Study (PISA YAFS), 2016. 


Exhibit 8. Subgroup differences in the strength of correlation between PISA 2012 and ESO scores from the U.S. PISA 
YAFS population overall, by subgroup and subjects: 2012 and 2016 


Subgroups with statistically significantly Correlation between Correlation between PISA 
different strength of correlation from ed bey WA' ey: Lel late ir: Lare| 2012 mathematics and 
U.S. PISA YAFS population overall STOR K-Yer- leave AUM Ls) ESO numeracy in 2016 
U.S. PISA YAFS population overall r=0.59 r=0.69 

White r=0.55 r=0.65 V 
Asian r=0.69 r=0.79 A 

From schools in villages (fewer than 3,000 people) r=0.49 r=0.54 V 

From schools in cities (between 100,000 and r=0.64 r=0.76 A 
1,000,000 people) 

From schools with less than 25 percent of students r=0.50 V r=0.63 V 
receiving free or reduced-price lunch 

From schools with 75 percent or more of students r=0.48 V r=0.54 V 
receiving free or reduced-price lunch 

In bottom quarter of index on openness to problem solving r=0.53 r=0.60 V 

In second-to-bottom quarter of index on openness to r=0.52 r= 0.60 


problem solving 


A Correlation between PISA 2012 and ESO scores is stronger for the subgroup than for the PISA YAFS population overall. 


V Correlation between PISA 2012 and ESO scores is weaker for the subgroup than for the PISA YAFS population overall. 

NOTE: ESO = Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in fall 2012 and who participated 
in PISA YAFS in 2016. Subgroups for which the strength of correlation between PISA 2012 and ESO scores is not statistically significantly different from 
the PISA YAFS population overall are not shown. Data for students eligible for free or reduced-price lunch were available for public schools only. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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4. Transitions From High 
School to Postsecondary Life 


This section examines U.S. 19-year-olds’ transitions 
from high school to young adulthood—focusing on 

their transitions to (1) postsecondary education, (2) the 
workforce, and (3) adult life. For each of these areas, 
the report summarizes the paths pursued by the U.S. 
PISA YAFS population overall and then examines 
whether those paths differed based on their proficiency 
in PISA 2012 at age 15. The summaries are based on 

the percentages of U.S. 19-year-olds in the ESO literacy 
assessment, which closely mirror the percentages of 
U.S. 19-year-olds in the ESO numeracy assessment. The 
estimates and standard errors for both the ESO literacy 
and numeracy populations can be found in appendix A. 


Transitions to postsecondary 
education 


The variables examined for the transitions to 
postsecondary education come from some of the 
literature discussed in the Introduction. These variables 
include the degrees and areas of study being pursued 
as well as participation in nonformal education 
opportunities, and they provide information on the 
extent to which U.S. 19-year-olds of different skill levels 
are participating in education after high school. 


Degree currently pursued 


In 2016, when PISA YAFS was conducted, 85 percent of 
U.S. 19-year-olds had completed a high school diploma 
(table A-21 and figure 5). This includes 45 percent of 
19-year-olds who were pursuing a bachelor’s degree or 
higher, 20 percent who were pursuing a pre-associate’s or 
associate’s degree, and 20 percent who were not studying 
or pursuing any degree after attaining a high school 
diploma. About 15 percent of U.S. 19-year-olds had not 


completed a high school diploma at the time of the study: 
9 percent were still pursuing it and 6 percent had left 
school without attaining it (i.e., dropped out). 


The percentages of U.S. 19-year-olds pursuing these 
various educational paths often related to the proficiency 
they showed as 15-year-olds in PISA 2012. Among high 
performers in PISA 2012 reading (i.e., at level 5 or above), 
98 percent had completed a high school diploma by 
2016—with 84 percent pursuing a bachelor’s degree or 
higher, which was larger than the percentage of the 
population doing so overall (45 percent) (table A-21 and 
figure 5). Conversely, the percentage of high performers 
in PISA 2012 reading pursuing pre-associate’s or 
associate’s degrees (11 percent) at age 19 was smaller than 
in the population overall (20 percent). The same patterns 
held for high performers in PISA 2012 mathematics (table 
A-22 and figure 6). 


Among U.S. 19-year-olds who were low performers in 
PISA 2012 reading (i.e., below level 2), 59 percent had 
completed a high school diploma by 2016, which was 

a smaller percentage than in the population overall 

(85 percent) (table A-21 and figure 5). In addition, 
proportionately fewer low performers in PISA 2012 
reading than in the population overall were pursuing 

a bachelor’s degree or higher (12 vs. 45 percent). In 
contrast, proportionately more low performers in 

PISA 2012 reading were still pursuing their high school 
diploma (23 percent) at age 19 than the population overall 
(9 percent). The same patterns held for low performers in 
PISA 2012 mathematics (table A-22 and figure 6). 


There were no statistically significant differences in 
degrees pursued between middle performers in PISA 
2012 reading or mathematics (i.e., at levels 2-4) and the 
population overall (tables A-21 and A-22). 


This percentage is the sum of the percentages of U.S. 19-year-olds shown in table A-21 and figure 5 who were (1) pursuing pre-associate’s and 
associate’s degrees, (2) pursuing bachelor’s degrees and above, and (3) not currently studying but had a high school diploma. 


Figure 5. Percentage distribution of U.S. 19-year-olds, by PISA 2012 reading proficiency levels and degrees currently 
pursued: 2016 


Percent 
100 
84* 

80 

60 

40 

20 

0 tf LLL. LLL FAN ZZ 
U.S. 19-year-olds Low Middle High 
(All levels) (Below level 2) (Levels 2—4) (Level 5 and above) 
Proficiency level 
fi Bachelor's degree ® Pre-associate’s or [1 Highschool fH Not currently studying, A Not currently studying, does 
and above associate’s degree diploma has a high school diploma not have a high school diploma 


! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents pursuing the given degree is significantly different for this proficiency level than it is for the overall population (p < .05). 
NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. The 
education characteristics data in this figure are from a background questionnaire that they completed as part of PISA YAFS. Solid columns indicate U.S. 
19-year-olds who have completed a high school diploma. Striped columns indicate those who have not completed a high school diploma. There were no 
19-year-olds in a sixth category: not currently studying, degree unknown. See exhibit 2 for a description of proficiency levels. Detail may not sum to totals 
because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Figure 6. Percentage distribution of U.S. 19-year-olds, by PISA 2012 mathematics proficiency levels and degrees 
currently pursued: 2016 


Percent 
100 
86* 
80 
60 
45 
40 
2 
20 cs é 
SA te é; ral |e ae: 
U.S. 19-year-olds Low Middle High 
(All levels) (Below level 2) (Levels 2—4) (Level 5 and above) 
Proficiency level 
i Bachelor's degree MM Pre-associate’s or [1 Highschool { Not currently studying, ‘A Not currently studying, does 
and above associate’s degree diploma has a high school diploma not have a high school diploma 


! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

$+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents pursuing the given degree is significantly different for this proficiency level than it is for the overall population (p < .05). 
NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. The 
education characteristics data in this figure are from a background questionnaire that they completed as part of PISA YAFS. Solid columns indicate U.S. 
19-year-olds who have completed a high school diploma. Striped columns indicate those who have not completed a high school diploma. There were no 
19-year-olds in a sixth category: not currently studying, degree unknown. See exhibit 3 for a description of proficiency levels. Detail may not sum to totals 
because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Area of study currently pursued The area of study pursued by U.S. 19-year-olds varied 
by their proficiency in PISA 2012 at age 15. The high 


performers in PISA 2012 reading (i.e., at level 5 and 
above) more frequently pursued studies in science, 
engineering, and humanities than did the population 
overall (figure 7a). For example, 27 percent of high 
performers in reading were studying science, which 
was 15 percentage points higher than the percentage in 
that field in the population overall. Similarly, 15 percent 
of high performers were studying engineering, and 

14 percent were studying humanities; in each case, 


In 2016, about 65 percent of U.S. 19-year-olds were 
pursuing a degree, including pre-associate’s or associate’s 
degrees and bachelor’s or higher degrees (table A-21). The 
most common areas of study pursued by these 19-year- 
olds were social sciences (15 percent), health (13 percent), 
and the sciences (12 percent). Additionally, 7 percent of 
U.S. 19-year-olds were studying engineering, and 6 percent 
were studying the humanities. Five percent or less of U.S. 
19-year-olds were pursuing degrees in each of the areas of 
general studies, teaching, services, and agriculture. 


Figure 7. Percentage of U.S. 19-year-olds, by PISA 2012 reading proficiency levels and areas of study currently 
pursued: 2016 


Percent a. High performers (i.e., Level 5 and above) 
100 
80 
60 
40 
27* 26 
20 —15.18 13.4349 gue 1a 9 
6 | \* 
Social Health Sciences Engineering Human- General Teacher Services Agriculture Currently Not 
sciences ities programs _ training pursuing currently 
a high school studying 
@ U.S. 19-year-olds (All levels) | J High (Level 5 and above) diploma 
Percent b. Low performers (i.e., Below level 2) 
100 
80 
si 49* 
40 
23 26 
5!* 7 3! 6 5 5 2 4! : 
0 a t | i | | t | m ? mt mt — a # 
Social Health Sciences Engineering Human- General Teacher Services Agriculture Currently Not 
sciences ities programs _ training pursuing currently 
a high school studying 
@ U.S. 19-year-olds (All levels) Low (Below level 2) diploma 


# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents pursuing the given area of study is significantly different for this proficiency level than it is for the overall population (p < .05). 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. The education 
characteristics data in this figure are from a background questionnaire that they completed as part of PISA YAFS. See exhibit 2 for a description of proficiency 
levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and Program for 
International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


8 This section does not consider the percentages of 19-year-olds currently pursuing a high school diploma or those not currently studying. The former 
is equivalent to the percentage of 19-year-olds pursuing a high school diploma described in the prior section, and the latter is equivalent to earlier 
described percentages of 19-year-olds not currently studying, either with or without a high school diploma. 
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the percentage was 8 percentage points higher than pursuing postsecondary degrees at all: 28 percent of 


that in the population overall. Except for humanities, low performers were pursuing a postsecondary degree 
these patterns held for high performers in PISA 2012 at the time of the survey, compared to 65 percent of the 
mathematics as well (table A-22 and figure 8a). population overall (table A-21). 

The low performers in PISA 2012 reading (i.e., below This difference did not hold for low performers in PISA 
level 2) were less represented than the population overall 2012 mathematics, who differed from the population 

in one of the three reportable areas of study;'* 5 percent overall in other areas of study (some of which were not 
of low performers were studying health compared with reportable in reading) (table A-22 and figure 8b). Among 
13 percent in the population overall (figure 7b). This is low performers in mathematics, the rates of pursuing 
likely related to the lower incidence of low performers three of the seven reportable areas of study at age 19 were 


Figure 8. Percentage of U.S. 19-year-olds, by PISA 2012 mathematics proficiency levels and areas of study currently 
pursued: 2016 


Percent a. High performers (i.e., Leve 5 and above) 
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sciences ities programs _ training pursuing currently 
a high school studying 
@ U.S. 19-year-olds (All levels) Low (Below level 2) diploma 


# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

$ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents pursuing the given area of study is significantly different for this proficiency level than it is for the overall population (p < .05). 
NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. 
The education characteristics data in this figure are from a background questionnaire that they completed as part of PISA YAFS. See exhibit 3 for a 
description of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


4 “Reportable areas of study” focuses on the disciplines listed in tables A-21 and A-22 and figures 7a, 7b, 8a, and 8b, excluding 19-year-olds still 
pursuing a high school diploma or not currently studying. The number of reportable areas differs based on the subject and the proficiency level of 
19-year-olds, with nonreportable areas presented in the referenced tables and figures. 
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lower than in the population overall, including social 
sciences (6 vs. 15 percent), sciences (3 vs. 12 percent), 

and engineering (2 vs. 7 percent). There were no 
statistically significant differences in the areas of study 
pursued between middle performers in PISA 2012 reading 
or mathematics (i.e., those at proficiency levels 2-4) and 
the PISA YAFS population overall (tables A-21 and A-22). 


Nonformal education participation 
in the last 12 months 


In addition to formal education, U.S. 19-year-olds were 
asked about participation in nonformal education, 
defined as structured learning activities that take place 
outside the formal education system. Examples of 
nonformal education include courses conducted through 
open or distance education, organized sessions for on- 
the-job training or training by supervisors or coworkers, 
seminars or workshops, and other kinds of courses or 
private lessons. 


In 2016, in the literacy and numeracy populations, 

66 to 67 percent of U.S. 19-year-olds had participated in 
nonformal education in the prior 12 months (tables A-21 
and A-22), and about one-third (33 percent) had not. 
Unlike formal education, participation in nonformal 
types of education activities was not related to prior 
proficiency in PISA 2012 reading or mathematics. 


Transitions to the workforce 


The variables examined for the transitions to the 
workforce also come from some of the literature 
discussed in the Introduction. The first two variables 
examine employment status—the first without considering 
concurrent education status and the second considering 
the interaction of education and employment (i.e., 
combined education and employment status). The third 
variable examines occupation. Together, these variables 
provide insight into the extent to which U.S. 19-year-olds 
of different proficiency levels are participating in the 
workforce, and in which jobs, and how they might be 
balancing that with educational pursuits. 


Employment status 


Forty-three percent of U.S. 19-year-olds were unemployed 
in 2016, which includes both those who were not looking 
for work at the time of the study (24 percent) and those 
who were looking for work (18 percent) (table A-23 

and figure 9). Most commonly, U.S. 19-year-olds were 
employed part time (40 percent). About 13 percent were 
employed full time, 3 percent identified as having “other” 
employment status, and 1 percent were working as an 
apprentice/intern. 


Employment status varied by their proficiency in PISA 

at age 15, but there were fewer differences related to 
proficiency than with educational outcomes (degrees and 
areas of study pursued). Among U.S. 19-year-olds who 
were high performers at age 15 in PISA 2012 reading (i.e., 
at level 5 and above), 54 percent were unemployed at age 
19 (table A-23). The percentage of high performers who 
were unemployed but not looking for work (46 percent) 
was 22 percentage points higher than in the same 
category in the population overall, while the percentage 
who were unemployed and looking for work (8 percent) 
was lower by 10 percentage points (figure 9). Additionally, 
just 5 percent of PISA 2012 high performers in reading 
were employed full time, which was 8 percentage points 
lower than in the population overall. As discussed 

in the next section, these results may be related to 

high performers’ relatively higher rates of pursuit of 
postsecondary degrees, suggesting that these 19-year-olds 
were not working, or were not looking for work, because 
they were enrolled in postsecondary education (table 
A-21). The patterns held for high performers in PISA 2012 
mathematics as well (table A-24 and figure 10). 


There were no statistically significant differences related 
to employment status between low performers in PISA 
2012 reading or mathematics (i.e., below level 2) and 
the population overall (tables A-23 and A-24). Similarly, 
there were no statistically significant differences in 
employment status between middle performers in PISA 
2012 reading or mathematics (i.e., at levels 2-4) and the 
population overall (tables A-23 and A-24). 


' These variables are based on the data available from the PISA YAFS questionnaire. They may differ from other student and employment variables 


available from NCES (see, e.g., Provasnik 2018). 


Figure 9. Percentage distribution of U.S. 19-year-olds, by PISA 2012 reading proficiency levels and employment status: 
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! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

$ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents in the employment status category is significantly different for this proficiency level than it is for the overall population (p < .05). 
NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. 

The employment characteristics data in this figure are from a background questionnaire that they completed as part of PISA YAFS. U.S. 19-year-olds 
below level 2 and at level 5 and above had unreportable estimates for the “Apprentice/intern” and “Other” categories. See exhibit 2 for a description of 
proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Figure 10. Percentage distribution of U.S. 19-year-olds, by PISA 2012 mathematics proficiency levels and employment 
status: 2016 
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! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

$+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents in the employment status category is significantly different for this proficiency level than it is for the overall population 

(p < .05). 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. The 
employment characteristics data in this figure are from a background questionnaire that they completed as part of PISA YAFS. U.S. 19-year-olds below 
level 2 had unreportable estimates for the “Apprentice/intern” category and U.S. 19-year-olds at level 5 and above had unreportable estimates for the 
“Other” category. See exhibit 3 for a description of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Combined education and employment education only at age 19, which was 17 percentage 
points higher than the percentage in the population 


status overall (table A-23 and figure 11). Forty-three percent 

This section examines the interaction of education and of high performers in PISA 2012 reading were in both 

employment status. In 2016, about 35 percent of U.S. employment and formal education, which was not 

19-year-olds were in formal education only, while 16 statistically significantly different from the rate for 

percent were in employment only (table A-23). More the overall population (38 percent). These patterns in 

commonly, U.S. 19-year-olds were both in employment employment and education paths held for PISA 2012 

and formal education (38 percent). Eight percent were mathematics as well (table A-24 and figure 12). 

neither employed nor in formal education. For a small 

percentage of U.S. 19-year-olds (3 percent), employment Low performers in PISA 2012 reading (i.e., below 

status was unknown, which includes some 19-year-olds level 2) had a lower rate of combining employment and 

(1 percent) who were in formal education and some education at age 19 (19 percent) than the population 

(2 percent) who were not. overall (38 percent) (table A-23 and figure 11). This pattern 
held for PISA 2012 mathematics as well (table A-24 and 

As with participation in formal education, PISA figure 12). Additionally, low performers in mathematics 

performance at age 15 was related to outcomes at age 19. had a higher rate of employment only (28 percent) than 

The majority of high performers in PISA 2012 reading the population overall (16 percent). 


(i.e., at level 5 and above) (52 percent) were in formal 


Figure 11. | Percentage distribution of U.S. 19-year-olds, by PISA 2012 reading proficiency levels and combined 
employment and education status: 2016 
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# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents in the combined education and employment status category is significantly different for this proficiency level than it is for the 
overall population (p < .05). 

‘Employment is defined as being either full- or part-time employed or an apprentice/intern. 

? Employment status unknown indicates “other” was selected for the employment status. 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. 

The employment and education characteristics data in this figure are from a background questionnaire that they completed as part of PISA YAFS. 

U.S. 19-year-olds below level 2 had unreportable estimates for the “Not in formal education, employment status unknown’ category. U.S. 19-year- 

olds at level 5 and above had unreportable estimates in this category, as well as in the “In employment only,” “In formal education, employment status 
unknown,” and “Neither employed nor in formal education” categories. U.S. 19-year-olds at levels 2-4 had unreportable estimates for the “Not in formal 
education, employment status unknown” category. See exhibit 2 for a description of proficiency levels. Detail may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Figure 12. 
employment and education status: 2016 


Percentage distribution of U.S. 19-year-olds, by PISA 2012 mathematics proficiency levels and combined 
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$+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 
* Percentage of respondents in the combined education and employment status category is significantly different for this proficiency level than it is for the 


overall population (p < .05). 


‘ Employment is defined as being either full- or part-time employed or an apprentice/intern. 

? Employment status unknown indicates “other” was selected for the employment status. 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. 

The employment and education characteristics data in this figure are from a background questionnaire that they completed as part of PISA YAFS. 

U.S. 19-year-olds below level 2 had unreportable estimates for the “In formal education, employment status unknown” category. U.S. 19-year-olds at 
level 5 and above had unreportable estimates in this category, as well as in the “In employment only,” “Not in formal education, employment status 
unknown,” and “Neither employed nor in formal education” categories. U.S.19-year-olds at levels 2-4 had unreportable estimates for the “Not in formal 
education, employment status unknown’ category. See exhibit 3 for a description of proficiency levels. Detail may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


There were no statistically significant differences in 
employment status between middle performers in PISA 
2012 reading or mathematics (i.e., at levels 2-4) and the 
population overall (tables A-23 and A-24). 


Current occupation 


Fifty-seven percent of U.S. 19-year-olds were employed 

in 2016,'° and by far the most common occupation was 

as sales or service workers (31 percent) (table A-23). 

The remaining U.S. 19-year-olds who were employed 
were distributed in small percentages across the other 
occupations. Five percent were professionals; 4 percent 
were craft and related trades workers or were technicians 
and associate professionals; 3 percent were in elementary 
occupations, or were clerical support workers, or were 
managers; 2 percent were in the armed forces; and 

1 percent were in skilled agricultural, forestry, and fish. 


These rates of occupations generally did not vary by PISA 
2012 proficiency level, although not all categories were 
reportable for all proficiency levels and subjects. 


Among high performers in PISA 2012 reading at age 15 
(i.e., at level 5 and above), rates for the four reportable 
occupations—professionals, technicians and associate 
professionals, clerical support workers, and service and 
sales workers—were not statistically significantly different 
from those in the population overall at age 19 (table A-23). 
These patterns held for high performers in PISA 2012 
mathematics (table A-24). 


Among low performers in PISA 2012 reading (i.e., below 
level 2), rates for the two reportable occupations—service 
and sales workers, and craft and related trades workers— 
were not statistically significantly different at age 19 from 
those in the population overall (table A-23). In addition 


‘6 This percentage is the sum of percentages of all occupation categories. It excludes the percentages of U.S. 19-year-olds who were unemployed and 
looking for work (18 percent) or unemployed or not looking for work (24 percent). 
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to these two occupations, five others—professionals; 
technicians and associate professionals; clerical support 
workers; plant and machine operators, and assembly 
workers; and elementary occupations—were occupied 
by low performers in PISA 2012 mathematics at rates not 
statistically significantly different from the population 
overall (table A-24). 


There were no statistically significant differences in 
occupation between middle performers in PISA 2012 
reading or mathematics (i.e., at levels 2-4) and the 
population overall (tables A-23 and A-24). 


Transitions to adult life 


Transitions to adult life are another important area 

to study to provide a broader picture than the more 
frequently studied educational and workforce paths. 
Whereas the variables examined in the prior subsections 
focused on self-reports of behavior or conditions, the 
variables in this section focus on self-reports of beliefs, 
attitudes, and interests. While affective measures are 
not new or unstudied, they have not always been the 
focus of youth transition studies and thus provide 
supplemental information about U.S. 19-year-olds’ 
choices and behaviors. However, it should be noted that, 
perhaps even more than the education and employment 
variables, these affective variables may be susceptible to 
change with maturity, and readers should remember that 
the current results are captured at just one point in time. 


Level of self-efficacy toward job- 
seeking 


Self-efficacy is the extent or strength of one’s belief in 
one’s own ability to complete tasks and achieve goals 
(KankaraS 2017). ESO focused specifically on self- 
efficacy toward job-seeking and asked individuals who 
were employed or looking for work” to respond to four 
related questions, featuring a 6-point scale ranging from 
“strongly agree” to “strongly disagree.” Each individual’s 
responses were then averaged, which allowed them to be 
placed in one of three categories of self-efficacy toward 
job-seeking: low, moderate, or high (OECD 2015). 


In 2016, there were more U.S. 19-year-olds who reported 
moderate self-efficacy toward job-seeking (39 percent) 
than other levels, with 22 percent having high and 

8 percent having low self-efficacy toward job-seeking 
(table A-25). About one-quarter (24 percent) of U.S. 


19-year-olds were not asked the survey questions 
(because they were unemployed and not looking for 
work), and 6 percent did not respond to the questions. 
These patterns did not vary by PISA 2012 proficiency 
(except for high performers, who had higher rates of not 
being asked the survey questions). 


Unlike most other variables examined in the study, there 
were no statistically significant differences in levels of 
self-efficacy toward job-seeking for either high or low 
performers in PISA 2012 reading or mathematics and the 
population overall (tables A-25 and A-26). Similarly, there 
were no statistically significant differences in levels of 
job-seeking self-efficacy for middle performers in PISA 
2012 reading or mathematics (i.e., levels 2-4) and the 
population overall (tables A-25 and A-26). 


Life satisfaction 


Life satisfaction describes individuals’ satisfaction with 
their lives as a whole and is considered to be an outcome 
that contributes to overall well-being (Diener et al. 1985). 
In ESO, individuals were asked to respond to four related 
questions on a 6-point scale (ranging from “strongly 
agree” to “strongly disagree”), which allowed them to be 
placed in one of three categories of life satisfaction: high, 
moderate, or low (OECD 2015). 


In 2016, at least two-thirds of U.S. 19-year-olds reported 
either moderate or high life satisfaction (37 and 34 percent, 
respectively) (table A-25).'° About one-fifth of U.S. 19-year- 
olds (21 percent) reported low levels of life satisfaction, and 
9 percent did not respond to the related survey questions. 
These percentages did not vary by PISA 2012 proficiency. 


Like with self-efficacy, there were no statistically 
significant differences in levels of life satisfaction 
between any level of performers in PISA 2012 reading 
or mathematics and the population overall (tables A-25 
and A-26). 


Vocational interests 


Individuals’ vocational interests are thought to reflect not 
only the types of work they would enjoy but personality 
characteristics as well (Kankaras 2017). Moreover, 
vocational interests have been found to influence 
cognitive competencies and, when well matched with 
individuals’ educational and occupational choices, their 
well-being and satisfaction (Kankaras 2017). In ESO, 
individuals were asked to respond, on a 5-point scale, 


Because the ESO background questionnaire’s battery of questions on self-efficacy were related to job-seeking, U.S. 19-year-olds who were 


unemployed and not looking for work were routed out of this question. 


8 Like self-efficacy, life-satisfaction categories are based on 19-year-olds’ responses to related survey items. 


to 60 related questions about the degree to which they 
would like or dislike various activities. This was based 
on the short-form version of the O*NET Interest Profile 
assessing the Holland RIASEC model of vocational 
interest (Holland 1997; National Center for O*NET 
Development n.d.; Rounds et al. 2010). There were 

10 questions about different types of work activities in 
each of six dimensions: realistic, investigative, artistic, 
social, enterprising, and conventional (see the following 
text box and OECD 2015). Individuals’ responses to the 
questions were added for each dimension and ranged 
from O to 40. The higher the score, the higher the 
interest in vocations in the given dimension. This section 
presents these index scores averaged across the PISA 
YAFS population. 


In 2016, U.S. 19-year-olds were most interested in social 
vocations, or those that involve working with other 
people to help them learn and grow; they scored higher 
for interest in these vocations (21) than any other type 
(table A-25). U.S. 19-year-olds scored lowest for interest in 
realistic or conventional vocations (15 each). There were 
some differences in vocational interests related to PISA 
2012 proficiency. 


Defining vocational interests 


Vocational 
interest DY =x-Yoa tf od (ola) 


High performers in PISA 2012 reading at age 15 (i.e., 

at level 5 and above) had a higher level of interest in 
investigative vocations than the population overall 
(scoring 20 vs. 18, respectively) and a lower level of 
interest in enterprising vocations than the population 
overall (scoring 17 vs. 19, respectively) at age 19. 
Investigative vocations include work that involves ideas 
and thinking rather than physical activity or leading 
people. Enterprising vocations include work that involves 
starting up and carrying out business projects (table A-25 
and figure 13). These patterns were the same for high 
performers in mathematics (table A-26 and figure 14). 


Among low performers in PISA 2012 reading (i.e., 
below level 2), there were no statistically significant 
differences in vocational interests at age 19 compared 
to the population overall (table A-25 and figure 13). Low 
performers in PISA 2012 mathematics, however, had 
lower interest in investigative vocations (16) than the 
population overall (18) (table A-26 and figure 14). 


There were no statistically significant differences in 
the vocational interests of middle performers in either 
subject and the population overall (tables A-25 and A-26). 


People with realistic interests like work that includes practical, hands-on problems and answers. Often 


Bedlisne people with realistic interests do not like careers that involve paperwork or working closely with others. 
betas People with investigative interests like work that has to do with ideas and thinking rather than physical 
Investigative A : 
activity or leading people. 
‘Avice People with artistic interests like work that deals with the artistic side of things, such as acting, music, art, 
and design. 
S6cial People with social interests like working with others to help others learn and grow. They like working with 


people more than working with objects, machines, or information. 


Enterprising 


People with enterprising interests like work that has to do with starting up and carrying out business projects. 
They like taking action rather than thinking about things. 


Conventional 


People with conventional interests like work that follows set procedures and routines. They prefer working 


with information and paying attention to details rather than working with ideas. 


SOURCE: Education and Skills Online Technical Documentation (OECD 2015). 
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Figure 13. Average scores of U.S. 19-year-olds on index of vocational interest, by PISA 2012 reading proficiency levels 
and vocational interests: 2016 
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* Percentage of respondents in the vocational interest category is significantly different for this proficiency level than it is for the overall population (p < .05). 
NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. 

The self-reported vocational interest data in this figure are from a background questionnaire that they completed as part of PISA YAFS. The scores 

on the vocational scales range from 0 to 40. The realistic type of individual is interested in manual types of jobs, the investigative type is curious and 
scientifically oriented, the artistic type prefers creative activities through different art forms, the social type is interested in working with people, the 
enterprising type leads and influences people, and the conventional type prefers well-structured situations. For more information on the vocational scales 
and interest categories, see OECD 2015. See exhibit 2 for a description of proficiency levels. Detail may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Figure 14. Average scores of U.S. 19-year-olds on index of vocational interest, by PISA 2012 mathematics proficiency 
levels and vocational interests: 2016 
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* Percentage of respondents in the vocational interest category is significantly different for this proficiency level than it is for the overall population (jp < .05). 
NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. 

The self-reported vocational interest data in this figure are from a background questionnaire that they completed as part of PISA YAFS. The scores 

on the vocational scales range from 0 to 40. The realistic type of individual is interested in manual types of jobs, the investigative type is curious and 
scientifically oriented, the artistic type prefers creative activities through different art forms, the social type is interested in working with people, the 
enterprising type leads and influences people, and the conventional type prefers well-structured situations. For more information on the vocational scales 
and interest categories, see OECD 2015. See exhibit 3 for a description of proficiency levels. Detail may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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High performance on PISA 2012 reading and mathematics 
was associated with higher education trajectories and 
higher rates of other potentially advantageous outcomes 
(e.g., higher rates of enrollment in bachelor’s degree 
programs and higher uptake in critical fields, such as 
science and engineering’). Low performance on PISA 
2012 reading and mathematics was associated with lower 
education trajectories at age 19 (e.g., still pursuing a 

high school diploma) as well as lower rates of potentially 
advantageous outcomes (e.g., pursuing a bachelor’s 
degree or higher or combining work and education). 


Looking across education, 
employment, and adult life 
outcomes 


Looking across education, employment, and adult life 
outcomes, it does appear that proficiency in PISA 2012 
at age 15—in particular, high performance on either the 
reading or mathematics scale—is related to many of the 
education and employment outcomes and one of the 
adult life outcomes at age 19. Exhibit 9 summarizes the 
outcomes that are more, or less, commonly attained 

for high performers (i.e., level 5 and above) and for low 
performers (i.e., below level 2) on the PISA 2012 reading 
and mathematics scales. 


Altogether, the picture of both high and low performers 
and their transition to adult life is substantially different 
than it is for the overall population on the variables 
studied in this report. 


Exhibit 9. Education, employment, and life outcomes for U.S. 19-year-old high and low performers in PISA 2012 relative 
to the PISA YAFS population overall: 2016 


WY Col e=Mexe)nalanreyamexeyin] oy-1x:1e Mm Coll ad boyeN Less common compared to PISA 
ed be AWANM WN oX-V a Colanit-laler-me | Kole e) YAFS population overall in 2016 YAFS population overall in 2016 


High performers in PISA 2012 ° Pursue a bachelor’s degree or ° Pursue pre-associate’s or 


reading and mathematics 
(Level 5 or above) 


higher 

Study sciences or engineering 
Study humanities’ 

Are unemployed but not looking 
for work 

Are in formal education only 
Are interested in investigative 
vocations 


associate’s degree 

Work full time 

Are unemployed and looking 
for work 

Are interested in enterprising 
vocations 


Low performers in PISA 2012 
reading and mathematics 
(Below level 2) 


Are still pursuing a high school 
diploma 
Are in employment only? 


Pursue a bachelor’s degree 

or higher 

Study health’ 

Study social science, sciences, 
or engineering? 

Are in both employment and 
education 

Are interested in investigative 
vocations? 


' This outcome is statistically significant for reading only. 
? This outcome is statistically significant for mathematics only. 
NOTE: Unless otherwise noted, all outcomes in the table are statistically significant for both reading and mathematics. Estimates for 19-year-olds 


are for individuals who were 15-year-old students in fall 2012 and who participated in PISA YAFS in 2016. High performers in PISA 2012 are those at 
proficiency level 5 and above; low performers are those below proficiency level 2. This table does not include outcomes for middle performers in PISA 
2012 (levels 2-4) because there were no statistically significant differences between them and the population overall in either PISA 2012 reading or 
mathematics. See tables A-21 to A-26 for related data. See exhibits 2 and 3 for descriptions of proficiency levels. See appendix B for additional detail on 
vocational interests. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


18 See Brown et al. (2018) for a brief discussion of the importance of mathematics and science in preparing students for the current and future 
economy. See also U.S. Department of Education (2018, 2020). 
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5. Conclusion 


PISA YAFS provided an opportunity to examine U.S. 
performance at two points in time: at 15 years old (using 
assessment results from PISA 2012) and again at 19 years 
old (using assessment results from ESO, administered 

in 2016 in PISA YAFS). This report focused on several 
key questions: (1) how did U.S. 19-year-olds perform 

in ESO; (2) how did proficiency in ESO at age 19 relate 

to proficiency in PISA 2012 at age 15; (3) is there a 
correlation between ESO scores at age 19 and PISA 2012 
scores at age 15; and (4) were education, workforce, and 
life outcomes at age 19 related to proficiency in PISA 2012 
at age 15? 


Performance in ESO in 2016. In 2016, U.S. 19-year- 
olds had an average score of 266 points in ESO literacy 
and 260 points in ESO numeracy. In both subjects, 
most 19-year-olds performed at the middle level of 
proficiency—72 percent in literacy and 68 percent in 
numeracy. About one-fifth to one-quarter performed 
at the low level, and small percentages (10 and 

7 percent, respectively) performed at the high level. 
There was one difference in the distributions across 
proficiency levels by race/ethnicity in numeracy, 

but no statistically significant differences by sex, 
language spoken at home, or whether they were born 
in the United States for either subject. This suggests 
that most U.S. 19-year-olds, largely regardless of 
background, have literacy and numeracy skills that 
are above the baseline needed for further learning 
and full participation in society, albeit below those 
needed for handling more abstract, complex concepts 
and situations. 


Relating proficiency in ESO at age 19 to 
proficiency in PISA 2012 at age 15. Proficiency 

at age 19 was most closely related to proficiency 

at age 15 among the middle performers, who were 
the largest group in the PISA YAFS population at 
both ages. Seventy-seven percent of U.S. 19-year- 
olds performed at the middle level of proficiency in 
PISA 2012 reading at age 15, and 78 percent of them 
also performed at the middle level of proficiency in 
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ESO literacy at age 19. In mathematics, 68 percent 

of U.S. 19-year-olds performed at the middle level of 
proficiency in PISA 2012, and 79 percent of them also 
performed at the middle level of proficiency in ESO 
numeracy at age 19. These patterns held true for all 
subgroups except those individuals from the most 
economically challenged schools at age 15, based 

on free or reduced-price lunch (FRPL) status. For 
this subgroup, and for high and low performers, the 
pattern of remaining in the same-labeled proficiency 
level categories 4 years later was not as stark. 


Correlations between ESO scores at age 19 
and PISA 2012 scores at age 15. The correlation 
between PISA 2012 reading and ESO literacy was 
0.59 and between PISA 2012 mathematics and ESO 
numeracy was 0.69, which indicate strong positive 
relationships. There were some differences in the 
strength of correlations for different subgroups of 
U.S. 19-year-olds. Most notably, correlations were 
weaker in both subjects for those 19-year-olds 

who had attended the most or least economically 
challenged schools (based on FRPL status) at age 15. 


Relationship of education, workforce, and life 
outcomes at age 19 to proficiency in PISA 2012 
at age 15. Proficiency in PISA 2012 was related to 
education, workforce, and life outcomes in five of 
nine areas examined in the study, the exceptions 
being participation in nonformal education, current 
occupation, self-efficacy, and life satisfaction. For 
example, high reading and mathematics proficiency 
at age 15 was associated with higher education 
trajectories and other potentially advantageous 

life outcomes at age 19—including higher rates of 
enrollment in bachelor’s degree programs and higher 
uptake in science and engineering, two fields that 
policymakers and researchers generally consider 
critical to meeting the workplace and problem- 
solving demands of the future). Low reading and 
mathematics proficiency at age 15 was associated 
with lower education trajectories (e.g., being in 


high school at age 19) and lower rates of potentially 
advantageous outcomes (e.g., pursuing a bachelor’s 
degree or higher or combining work and education). 


Together, these results provide a snapshot of the 
cognitive skills of U.S. 19-year-olds and an analysis of how 
their skills, outcomes, attitudes, and interests are related 
to the academic proficiency they demonstrated at age 

15. Some of the results confirm previously uncovered 
relationships (e.g., between high performance and the 
pursuit of higher education or certain areas of study), 
while others reveal more novel findings (e.g., between 
performance and vocational interests). 
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Overall, by documenting generally strong, positive 
relationships between individuals’ reading and 
mathematics performance at age 15 and their literacy 
skills, numeracy skills, and educational trajectories 

4 years later, this report provides an indication of the 
degree to which success on PISA 2012 is related to various 
outcomes at the important transition from high school 
and the outset of adult life. However, it also showed that 
these relationships are not universal for all students, 
highlighting students who were in the poorest schools at 
age 15 as an at-risk group for skill loss. Further research 
might build on this development work and explore these 
findings in more detail. 
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Appendix A. Data Tables 


Table A-1. | Mean, standard deviation, and range of U.S. 19-year-olds’ ESO literacy assessment scores: 2016 


Standard 
ro f=\VAE- LiKe) a} Maximum score 


[Standard errors appear in parentheses] 
Total 266.1 (1.95) | 49.0 (1.35) | 79.9 (16.90) | 418.6 (14.09) 


NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Detail may not sum to totals because of 
rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-2. — Percentage distribution of U.S. 19-year-olds, by ESO literacy proficiency levels and selected demographic 


ESO literacy proficiency levels 
Total 


characteristics: 2016 


Low Middle High (Level 4 
(EXEL oA) (URW Gs) Elacemslerey 4:9) 


Demographic characteristics [Standard errors appear in parentheses] 
Total 100.0 (t) 18.8 (1.55) 71.7 (1.56) 9.5 (0.90) 
Gender 
Male 51.1 (1.56) 22.8 (2.46) 67.8 (2.32) 9.4 (1.10) 
Female 48.9 (1.56) 14.6 (1.80) 75.8 (2.11) 9.6 (1.16) 
Race/ethnicity 
White 52.5 (2.53) 14.5 (2.03) 73.0 (1.97) 12.5 (1.23) 
Black or African American 12.3. (1.72) 33.0 (6.74) 65.1 (6.75) t (t) 
Hispanic or Latino 23.0 (1.75) 21.7 (3.14) 72.4 (3.13) 5.9 (1.23) 
Asian 5.8 (1.28) 15.4! (6.77) 71.6 (6.71) 13.0 (3.70) 
Other 6.4 (1.07) 17.0 (3.87) 72.7 = (4.42) 10.3! (3.18) 
Born in the United States 
Native 77.1 (2.36) 17.8 (1.88) 71.8 (1.99) 10.4 (1.03) 
First-generation native 15.2 (1.64) 15.7 (2.83) 76.9 (2.95) 7.4 (2.18) 
Nonnative 7.7 (1.06) 24.2 (6.42) 67.4 (5.98) 8.4! (2.74) 
Language at home 
English 86.4 (1.35) 17.8 (1.64) 71.9 (1.72) 10.3 (1.01) 
Spanish 9.4 (1.04) 22.9 (4.71) 71.8 (4.63) 5.3! (2.05) 
Other languages 4.2 (0.73) 28.4! (9.32) 64.2 (9.07) 7.4! (2.65) 


t Not applicable. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

$ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Race categories exclude persons of 
Hispanic ethnicity. “Other” includes those who identified themselves as “Two or more races,” “American Indian/Alaska Native,” and “Native Hawaiian/ 
Other Pacific Islander.” See exhibit 4 for descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-3. | Mean, standard deviation, and range of U.S. 19-year-olds’ ESO numeracy assessment score: 2016 


Standard 
deviation Maximum score 


[Standard errors appear in parentheses] 
Total 259.5 (2.13) | 49.1 (1.34) | 91.4 (19.83) | 419.2 (13.04) 


NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Detail may not sum to totals because of 
rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-4. — Percentage distribution of U.S. 19-year-olds, by ESO numeracy proficiency levels and selected 


ESO numeracy proficiency levels 


Low Middle High (Level 4 
eel (EXEL CoA) (URW Gs) Elacemeslereh 4:9) 


demographic characteristics: 2016 


Demographic characteristics [Standard errors appear in parentheses] 
Total 100.0 (t) 24.5 (1.88) 68.2 (1.79) 7.3 (0.77) 
Gender 
Male 51.3 (1.59) 26.8 (2.74) 64.6 (2.63) 8.6 (1.12) 
Female 48.7 (1.59) 22.1 (2.06) 71.9 (1.93) 6.0 (0.94) 
Race/ethnicity 
White 52.6 (2.57) 17.0 (2.04) 72.9 (2.19) 10.0 (1.11) 
Black or African American 12.5 (1.71) 44.6* (6.16) 54.5 (6.13) t (t) 
Hispanic or Latino 22.9 (1.72) 31.0 (3.54) 65.2 (3.44) 3.8 (1.12) 
Asian 5.8 (1.29) 16.3! (6.89) 71.3 (6.28) 12.4 (3.55) 
Other 6.1 (1.05) 30.9 (8.57) 63.1 (8.21) 6.0! (2.24) 
Born in the United States 
Native 77.1 (2.30) 23.4 (2.16) 69.0 (2.15) 7.7 (0.85) 
First-generation native 15.1 (1.60) 26.3 (3.65) 67.4 (3.41) 6.3 (1.64) 
Nonnative 7.8 (1.02) 25.8 (7.41) 65.8 (7.22) 8.4! (2.65) 
Language at home 
English 86.3 (1.30) 22.9 (1.91) 69.4 (1.91) 7.7 (0.84) 
Spanish 9.4 (1.00) 33.4 (4.86) 62.2 (4.71) 44! (1.81) 
Other languages 4.2 (0.73) 27.0! (9.89) 62.7 (9.73) 10.2! (4.00) 


t Not applicable. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Subgroup percentage is statistically significantly different from the total percentage (p < .05). 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Race categories exclude persons of 
Hispanic ethnicity. “Other” includes those who identified themselves as “Two or more races,” “American Indian/Alaska Native,” and “Native Hawaiian/ 
Other Pacific Islander.” See exhibit 5 for descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-5. |= Percentage distribution of U.S. 19-year-olds, by ESO literacy proficiency level and PISA 2012 reading 
proficiency level: 2016 


ESO literacy proficiency levels 


Low Middle High (Level 4 
(CEXEN Co AEWZE1 74) (URW: i 2)) and above 


ed bY WAU AY: Le llare| 
proficiency levels 


[Standard errors appear in parentheses] 


Low (Below level 2) 51.3 (6.65) 48.5 (6.71) t (t) 
Middle (Levels 2—4) 15.0 (1.52) 77.6 (1.76) 7.4 (0.95) 
High (Level 5 and above) t (t) 57.4 (4.67) 41.3 (4.72) 


t Not applicable. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. See exhibits 2 and 4 for descriptions of 
proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-6. = Percentage distribution of U.S. 19-year-olds, by ESO literacy proficiency level, sex, and PISA 2012 reading 
proficiency level: 2016 


ESO literacy proficiency levels 


Low Middle High (Level 4 
CEZEN Co AEWZE1 4) (Levels 2—3) lake me loyeye 


ed bsY-WAUM et: Lellare | 
proficiency levels 


[Standard errors appear in parentheses] 


Male 
Low (Below level 2) 53.8 (7.57) 46.0 (7.63) t (t) 
Middle (Levels 2—4) 17.6 (2.50) 74.4 = (2.77) 8.0 (1.25) 
High (Level 5 and above) t (t) 51.2 (8.40) 47.7 (8.65) 
Female 
Low (Below level 2) 46.8 (12.76) 53.2 (12.76) # (t) 
Middle (Levels 2—4) 12.4 (1.84) 80.7 (2.38) 6.8 (1.09) 
High (Level 5 and above) t (t) 61.4 (4.42) 37.1 (4.65) 


t Not applicable. 
# Rounds to zero. 


+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 
NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. See exhibits 2 and 4 for descriptions of 

proficiency levels. Detail may not sum to totals because of rounding. 
SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 


Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-7. Percentage distribution of U.S. 19-year-olds, by ESO literacy proficiency level, national quarters of the PISA 
2012 index of economic, social, and cultural status (ESCS), and PISA 2012 reading proficiency level: 2016 


ESO literacy proficiency levels 


Low Middle High (Level 4 
ed Sy WAI WA Y-Le Tale] (Below level 2) (Levels 2-3) F-Vate ir- | ofenvi=) 


proficiency levels [Standard errors appear in parentheses] 


Bottom quarter 


Low (Below level 2) 59.0 (10.60) 41.0 (10.60) # (t) 

Middle (Levels 2—4) 20.4 (3.41) 74.7 (3.74) 4.9! (1.62) 

High (Level 5 and above) # (tT) 85.9 (15.86) t (Tt) 

Second quarter 

Low (Below level 2) 48.4 (13.71) 51.6 (13.71) # (t) 

Middle (Levels 2—4) 14.9 (2.54) 78.8 (3.16) 6.3 (1.62) 

High (Level 5 and above) # (t) 58.2 (11.81) 41.8 (11.81) 
Third quarter 

Low (Below level 2) 43.7! (14.12) 55.3 (14.33) t (t) 

Middle (Levels 2—4) 13.8 (3.42) 78.7 (3.73) 7.6 (1.79) 

High (Level 5 and above) t (t) 61.3 (9.33) 36.3 (9.00) 
Top quarter 

Low (Below level 2) 39.5! (15.42) 60.5 (15.42) # (T) 

Middle (Levels 2—4) 10.3 (2.29) 78.9 (2.75) 10.8 (2.27) 

High (Level 5 and above) t (tT) 52.9 (6.59) 46.0 (6.71) 


t Not applicable. 

# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. The PISA 2012 index of economic, 
social, and cultural status (ESCS) was created using student reports on parental occupation, the highest level of parental education, and an index of 
home possessions related to family wealth, home educational resources and possessions related to “classical” culture in the family home. The home 
possessions relating to “classical” culture in the family home included possessions such as works of classical literature, books of poetry, and works of 
art (e.g., paintings). See exhibits 2 and 4 for descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-8. | Percentage distribution of U.S. 19-year-olds, by ESO literacy proficiency level, free or reduced-price 
lunch (FRPL) status of students’ schools at age 15, and PISA 2012 reading proficiency level: 2016 


ESO literacy proficiency levels 


Low Middle High (Level 4 
ELAN) (Levels 2-3) Vale me loyeyye 


ed bY WAUM aa Y-Lellare | 
proficiency levels 


[Standard errors appear in parentheses] 


Schools with less than 25 percent of students eligible for FRPL 


Low (Below level 2) t (tT) 65.9! (21.69) t (t) 
Middle (Levels 2—4) 9.8 (1.78) 80.4 (2.02) 9.8 (1.29) 
High (Level 5 and above) t (tT) 55.4 (5.27) 43.1 (6.03) 


Schools with 25 to 49.9 percent of students eligible for FRPL 


Low (Below level 2) 51.7! (16.03) 48.3! (16.03) # (t) 
Middle (Levels 2—4) 12.8 (1.82) 76.9 (2.99) 10.3 (2.25) 
High (Level 5 and above) t (tT) 58.5 (10.72) 40.2 (10.81) 


Schools with 50 to 74.9 percent of students eligible for FRPL 


Low (Below level 2) 52.8 (9.94) 47.2 (9.94) # (t) 

Middle (Levels 2—4) 14.5 (2.55) 80.7 (2.53) 4.7 (1.37) 

High (Level 5 and above) t (t) 56.2 (10.99) 43.4 (11.10) 
Schools with 75 percent or more students eligible for FRPL 

Low (Below level 2) 55.8 (14.11) 44.2! (14.11) # (t) 

Middle (Levels 2—4) 28.5* (4.91) 67.8 (5.03) 3.7! (1.49) 

High (Level 5 and above) # (tT) 73.6 (16.37) t (tT) 


t Not applicable. 

# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

} Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Subgroup percentage is significantly different from the total percentage in table A-5 (p <.05). 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Data for students eligible for free or 
reduced-price lunch (FRPL) were available for public schools only. See exhibits 2 and 4 for descriptions of proficiency levels. Detail may not sum to totals 
because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-9. —- Percentage distribution of U.S. 19-year-olds, by ESO literacy proficiency level, race/ethnicity, and PISA 
2012 reading proficiency level: 2016 


ESO literacy proficiency levels 


Low Middle High (Level 4 
ed Sy WAI WAY Le Tale] (Below level 2) (Levels 2-3) and above 


proficiency levels [Standard errors appear in parentheses] 


White 
Low (Below level 2) 50.2 (14.81) 49.4! (15.21) t (t) 
Middle (Levels 2—4) 12.6 (2.36) 77.9 (2.89) 9.5 (1.51) 
High (Level 5 and above) t (tT) 56.4 (6.05) 42.3 (5.86) 
Black or African American 
Low (Below level 2) 60.4 (12.81) 39.6! (12.81) # (T) 
Middle (Levels 2—4) 22.7 (5.52) 75.0 (5.98) t (t) 
High (Level 5 and above) # (tT) 90.9 (17.49) t (t) 
Hispanic or Latino 
Low (Below level 2) 50.1! (15.61) 49.9! (15.61) # (t) 
Middle (Levels 2—4) 16.5 (3.13) 78.1 (3.90) 5.4 (1.57) 
High (Level 5 and above) t (t) 64.3 (12.69) 33.6! (12.57) 
Asian 
Low (Below level 2) 69.2! (31.64) t (tT) # (t) 
Middle (Levels 2—4) 10.5! (4.63) 81.9 (5.59) 7.7! (3.20) 
High (Level 5 and above) t (t) 52.9 (14.94) 46.0! (14.66) 
Other 
Low (Below level 2) t (Tt) 79.7 (16.67) # (t) 
Middle (Levels 2—4) 17.7 (4.97) 75.6 (5.08) 6.8! (2.64) 
High (Level 5 and above) t (tT) 48.5! (15.09) 50.8 (15.22) 


t Not applicable. 

# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Race categories exclude persons of 
Hispanic ethnicity. “Other” includes those who identified themselves as “Two or more races,” “American Indian/Alaska Native,” and “Native Hawaiian/ 
Other Pacific Islander.” See exhibits 2 and 4 for descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-10. = Percentage distribution of U.S. 19-year-olds, by ESO numeracy proficiency level and PISA 2012 
mathematics proficiency level: 2016 


ESO numeracy proficiency levels 


Low Middle High (Level 4 
PISA 2012 mathematics (Below level 2) (URW) 2) F-Tave i=] efe)vi:1 


proficiency levels [Standard errors appear in parentheses] 


Low (Below level 2) 60.1 (4.28) 39.6 (4.31) t (tT) 
Middle (Levels 2—4) 16.6 (1.99) 79.2 (2.00) 4.2 (0.59) 
High (Level 5 and above) t (t) 53.6 (3.76) 45.8 (3.87) 


t Not applicable. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. See exhibits 3 and 5 for descriptions of 
proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-11. Percentage distribution of U.S. 19-year-olds, by ESO numeracy proficiency level, sex, and PISA 2012 
mathematics proficiency level: 2016 


ESO numeracy proficiency levels 


Low Middle High (Level 4 
PISA 2012 mathematics (Below level 2) (URW) )) and above 


proficiency levels [Standard errors appear in parentheses] 


Male 
Low (Below level 2) 63.8 (5.88) 35.6 (5.92) t (tT) 
Middle (Levels 2—4) 18.8 (3.25) 76.0 (3.27) 5.2 (0.89) 
High (Level 5 and above) t (Tt) 49.4 (7.18) 49.6 (6.80) 
Female 
Low (Below level 2) 56.0 (5.79) 44.0 (5.79) # (tT) 
Middle (Levels 2—4) 14.3 (1.89) 82.5 (2.12) 3.2 (0.77) 
High (Level 5 and above) t (t) 58.3 (6.11) 41.5 (6.09) 


t Not applicable. 

# Rounds to zero. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. See exhibits 3 and 5 for descriptions of 
proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


48 


Table A-12. — Percentage distribution of U.S. 19-year-olds, by ESO numeracy proficiency level, national quarters of the 
PISA 2012 index of economic, social, and cultural status (ESCS), and PISA 2012 mathematics proficiency 
level: 2016 


ESO numeracy proficiency levels 


Low Middle High (Level 4 
PISA 2012 mathematics (Below level 2) (UR W(=) ye) F-Vale r-| ofeyv(:) 


proficiency levels [Standard errors appear in parentheses] 


Bottom quarter 


Low (Below level 2) 68.6 (7.01) 31.2 (7.16) t (t) 
Middle (Levels 2—4) 23.8 (5.84) 73.1 (5.90) 3.1! (1.23) 
High (Level 5 and above) t (t) 54.6! (16.54) 44.2! (16.89) 
Second quarter 
Low (Below level 2) 54.9 (7.90) 45.1 (7.90) # (tT) 
Middle (Levels 2—4) 16.9 (2.65) 79.6 (3.13) 3.5! (1.40) 
High (Level 5 and above) t (t) 56.1 (11.21) 42.6 (12.11) 
Third quarter 
Low (Below level 2) 55.5 (9.42) 43.2 (9.39) t (t) 
Middle (Levels 2—4) 16.7 (2.98) 78.3 (3.38) 5.0 (1.27) 
High (Level 5 and above) # (tT) 55.2 (9.20) 44.8 (9.20) 
Top quarter 
Low (Below level 2) 49.5 (12.57) 50.5 (12.57) # (t) 
Middle (Levels 2—4) 10.0 (2.26) 85.1 (2.11) 4.9 (1.36) 
High (Level 5 and above) t (t) 52.2 (5.32) 47.2 (5.67) 


t Not applicable. 

# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. The PISA 2012 index of economic, 
social, and cultural status (ESCS) was created using student reports on parental occupation, the highest level of parental education, and an index of 
home possessions related to family wealth, home educational resources and possessions related to “classical” culture in the family home. The home 
possessions relating to “classical” culture in the family home included possessions such as works of classical literature, books of poetry, and works of art 
(e.g., paintings). See exhibits 3 and 5 for descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-13. = Percentage distribution of U.S. 19-year-olds, by ESO numeracy proficiency level, free or reduced-price 
lunch (FRPL) status of students’ schools at age 15, and PISA 2012 mathematics proficiency level: 2016 


ESO numeracy proficiency levels 


Low Middle High (Level 4 
PISA 2012 mathematics (Below level 2) (Levels 2-3) FVale r-] ofeyv(=) 


proficiency levels [Standard errors appear in parentheses] 


Schools with less than 25 percent of students eligible for FRPL 


Low (Below level 2) 42.6! (14.27) 57.4 (14.27) # (t) 

Middle (Levels 2—4) 11.2 (2.94) 84.1 (2.72) 4.7 (1.18) 

High (Level 5 and above) t (t) 52.1 (5.33) 47.6 (5.58) 
Schools with 25 to 49.9 percent of students eligible for FRPL 

Low (Below level 2) 55.5 (12.09) 44.1 (12.03) t (t) 

Middle (Levels 2—4) 14.9 (2.83) 79.8 (3.03) 5.3 (1.10) 

High (Level 5 and above) t (t) 52.4 (7.14) 47.1 (6.87) 
Schools with 50 to 74.9 percent of students eligible for FRPL 

Low (Below level 2) 61.3 (6.33) 38.7 (6.33) # (tT) 

Middle (Levels 2—4) 14.1 (2.78) 82.3 (2.90) 3.6! (1.17) 

High (Level 5 and above) t (tT) 55.3 (10.59) 42.7 (9.97) 
Schools with 75 percent or more students eligible for FRPL 

Low (Below level 2) 67.3 (8.63) 31.9 (8.28) t (t) 

Middle (Levels 2—4) 35.5* (6.50) 62.1* (6.70) t (t) 

High (Level 5 and above) # (Tt) 65.3 (13.56) 34.7! (13.56) 


t Not applicable. 

# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Subgroup percentage is significantly different from the total percentage in table A-10 (p <.05). 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Data for students eligible for free or 
reduced-price lunch (FRPL) were available for public schools only. See exhibits 3 and 5 for descriptions of proficiency levels. Detail may not sum to totals 
because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-14. | Percentage distribution of U.S. 19-year-olds, by ESO numeracy proficiency level, race/ethnicity, and PISA 
2012 mathematics proficiency level: 2016 


ESO numeracy proficiency levels 


Low Middle High (Level 4 
EAN EL) (Levels 2-3) and above 


PISA 2012 mathematics 
proficiency levels 


[Standard errors appear in parentheses] 


White 
Low (Below level 2) 50.2 (9.80) 48.8 (9.86) t (t) 
Middle (Levels 2—4) 14.2 (2.57) 80.7 (2.73) 5.0 (1.01) 
High (Level 5 and above) t (t) 51.0 (4.26) 48.2 (4.53) 
Black or African American 
Low (Below level 2) 69.8 (7.43) 30.2 (7.43) # (t) 
Middle (Levels 2—4) 26.1! (8.96) 72.4 (9.01) t (t) 
High (Level 5 and above) # (t) 85.6! (27.70) t (t) 
Hispanic or Latino 
Low (Below level 2) 61.6 (7.36) 38.4 (7.36) # (t) 
Middle (Levels 2—4) 18.5 (3.17) 78.7 (3.33) 2.8! (1.01) 
High (Level 5 and above) # (t) 54.0 (9.29) 46.0 (9.29) 
Asian 
Low (Below level 2) 67.0! (27.93) t (tT) # (t) 
Middle (Levels 2—4) 12.6! (5.32) 80.6 (5.92) 6.9! (3.12) 
High (Level 5 and above) # (t) 62.1 (11.43) 37.9! (11.43) 
Other 
Low (Below level 2) 59.4 (14.97) 40.6! (14.97) # (T) 
Middle (Levels 2—4) 20.2! (6.75) 76.7 (7.12) t (t) 
High (Level 5 and above) # (t) 60.3 (14.44) 39.7! (14.44) 


t Not applicable. 
# Rounds to zero. 


! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 


$ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 
NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Race categories exclude persons of 
Hispanic ethnicity. “Other” includes those who identified themselves as “Two or more races,” “American Indian/Alaska Native,” and “Native Hawaiian/ 


Other Pacific Islander.” See exhibits 3 and 5 for descriptions of proficiency levels. Detail may not sum to totals because of rounding. 


SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-15. Percentage distribution of U.S. 19-year-olds and correlations between PISA 2012 reading scores and 
ESO literacy scores, by selected demographic characteristics: 2016 


Correlations between PISA 
2012 reading scores and ESO 


literacy scores in 2016 


Demographic characteristics [Standard errors appear in parentheses] 


Total 100.0 (t) 0.59 (0.025) 
Gender 

Male 51.1 (1.56) 0.59 (0.032) 

Female 48.9 (1.56) 0.57 (0.034) 
Race/ethnicity 

White 52.5 (2.53) 0.55 (0.035) 

Black or African American 12.3 (1.72) 0.50 (0.071) 

Hispanic or Latino 23.0 (1.75) 0.56 (0.044) 

Asian 5.8 (1.28) 0.69 (0.129) 

Other 6.4 (1.07) 0.54 (0.095) 
Born in the United States 

Native 77.1 (2.36) 0.57 (0.030) 

First-generation native 15.2 (1.64) 0.61 (0.050) 

Nonnative 7.7 (1.06) 0.64 (0.095) 
Language at home 

English 86.4 (1.35) 0.58 (0.028) 

Spanish 9.4 (1.04) 0.58 (0.071) 

Other languages 4.2 (0.73) 0.60 (0.151) 


t Not applicable. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Race categories exclude persons of 
Hispanic ethnicity. “Other” includes those who identified themselves as “Two or more races,” “American Indian/Alaska Native,” and “Native Hawaiian/ 
Other Pacific Islander.” Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-16. Percentage distribution of U.S. 19-year-olds and correlations between PISA 2012 mathematics scores and 
ESO numeracy scores, by selected demographic characteristics: 2016 


Correlations between PISA 
2012 mathematics scores and 


Total | ESO numeracy scores in 2016 


Demographic characteristics [Standard errors appear in parentheses] 


Total 100.0 (t) 0.69 (0.018) 

Gender 

Male 51.3 (1.59) 0.69 (0.022) 

Female 48.7 (1.59) 0.69 (0.026) 
Race/ethnicity 

White 52.6 (2.57) 0.65* (0.031) 

Black or African American 12.5 (1.71) 0.60 (0.064) 

Hispanic or Latino 22.9 (1.72) 0.67 (0.030) 

Asian 5.8 (1.29) 0.79* (0.092) 

Other 6.1. (1.05) 0.69 (0.058) 
Born in the United States 

Native 77.1 (2.30) 0.68 (0.022) 

First-generation native 15.1 (1.60) 0.72 (0.034) 

Nonnative 7.8 (1.02) 0.75 (0.055) 
Language at home 

English 86.3 (1.30) 0.69 (0.021) 

Spanish 9.4 (1.00) 0.70 (0.044) 

Other languages 4.2 (0.73) 0.76 (0.104) 


t Not applicable. 

* The correlation statistic for the subgroup is significantly different from the correlation statistic for the total (p < .05). 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 
and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Race categories exclude persons of 
Hispanic ethnicity. “Other” includes those who identified themselves as “Two or more races,” “American Indian/Alaska Native,” and “Native Hawaiian/ 
Other Pacific Islander.” Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-17. Percentage distribution of U.S. 19-year-olds and correlations between PISA 2012 reading scores and 
ESO literacy scores, by selected socioeconomic characteristics: 2016 


Correlations between 
ed bY WAI Ae ¥- lel late B-yere) f-¥) 


and ESO literacy scores 


in 2016 
Socioeconomic characteristics 
Total 100.0 (t) 0.59 (0.025) 
Highest level of parental educational attainment 
High school and below 39.3 (1.98) 0.53 (0.047) 
Associate’s degree 14.9 (1.26) 0.60 (0.043) 
Bachelor’s degree or higher 45.7 (1.84) 0.59 (0.033) 
School location’ 
Village (fewer than 3,000 people) 10.8 (2.57) 0.49 (0.113) 
Small town (3,000 to about 15,000 people) 13.8 (3.41) 0.60 (0.057) 
Town (15,000 to about 100,000 people) 38.7 (4.56) 0.54 (0.033) 
City (100,000 to about 1,000,000 people) 26.4 (3.32) 0.64 (0.048) 
Large city (with over 1,000,000 people) 10.3! (3.18) 0.58 (0.079) 
School type 
Public 94.0 (1.05) 0.58 (0.026) 
Private 6.0 (1.05) 0.65 (0.079) 


Percentage of students eligible for free or 
reduced-price lunch? 


< 25% (least economically challenged) 27.4 (3.86) 0.50* (0.066) 
> = 25% but < 50% 26.2 (3.52) 0.58 (0.046) 
> = 50% but < 75% 29.8 (4.45) 0.58 (0.041) 
>= 75% to < 100% (most economically challenged) 16.5 (3.82) 0.48* (0.083) 
Index of economic, social, and cultural status® 
Bottom quarter 24.5 (1.80) 0.53 (0.061) 
Second quarter 24.1 (1.20) 0.55 (0.053) 
Third quarter 24.3 (1.19) 0.55 (0.054) 
Top quarter 27.1 (1.88) 0.55 (0.040) 


t Not applicable. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

* The correlation statistic for the subgroup is significantly different from the correlation statistic for the total (p < .05). 

' Categorization for school location is based on OECD definition and differs from the standard NCES categories. 

? Data for students eligible for free or reduced-price lunch (FRPL) were available for public schools only. 

3 The PISA 2012 index of economic, social, and cultural status (ESCS) was created using student reports on parental occupation, the highest level of 
parental education, and an index of home possessions related to family wealth, home educational resources and possessions related to “classical” 
culture in the family home. The home possessions relating to “classical” culture in the family home included possessions such as works of classical 
literature, books of poetry, and works of art (e.g., paintings). The cut points are calculated using the whole U.S. PISA 2012 sample. 

NOTE: ESO stands for Education and Skills Online. Results reported are weighted estimates for the PISA YAFS population, which is nationally 
representative of PISA 2012 students 4 years later at approximately age 19. Estimates for 19-year-olds are for individuals who were 15-year-old students 
in the fall of 2012 and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Detail may not sum 
to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-18. Percentage distribution of U.S. 19-year-olds and correlations between PISA 2012 mathematics scores and 
ESO numeracy scores, by selected socioeconomic characteristics: 2016 


Correlations between 
PISA 2012 mathematics 


scores and ESO numeracy 
scores in 2016 


SYoles rey=Yere) aveyanl (emegar-le-leat-y a l-ja (exc) [Standard errors appear in parentheses] 


Total 100.0 (t) 0.69 (0.018) 
Highest level of parental educational attainment 

High school and below 39.3 (2.01) 0.68 (0.027) 

Associate’s degree 15.1 (1.29) 0.67 (0.052) 

Bachelor’s degree or higher 45.6 (1.90) 0.67 (0.034) 
School location’ 

Village (fewer than 3,000 people) 10.6 (2.52) 0.54* (0.139) 

Small town (3,000 to about 15,000 people) 13.7 (3.35) 0.68 (0.045) 

Town (15,000 to about 100,000 people) 38.8 (4.56) 0.66 (0.027) 

City (100,000 to about 1,000,000 people) 26.5 (3.33) 0.76* (0.034) 

Large city (with over 1,000,000 people) 10.4! (3.22) 0.70 (0.066) 
School type 

Public 93.7 (1.11) 0.69 (0.019) 

Private 6.3 (1.11) 0.68 (0.077) 


Percentage of students eligible for free or 
reduced-price lunch? 


< 25% (least economically challenged) 27.5 (3.89) 0.63* (0.045) 
> = 25% but < 50% 26.0 (3.49) 0.70 (0.035) 
> = 50% but < 75% 29.7 (4.44) 0.69 (0.032) 
> = 75% to < 100% (most economically challenged) 16.7 (3.87) 0.54* (0.086) 
Index of economic, social, and cultural status® 
Bottom quarter 24.7 (1.82) 0.66 (0.044) 
Second quarter 23.9 (1.20) 0.64 (0.042) 
Third quarter 24.5 (1.22) 0.65 (0.037) 
Top quarter 26.9 (1.91) 0.68 (0.033) 


t Not applicable. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

* The correlation statistic for the subgroup is significantly different from the correlation statistic for the total (p < .05). 

' Categorization for school location is based on OECD definition and differs from the standard NCES categories. 

? Data for students eligible for free or reduced-price lunch (FRPL) were available for public schools only. 

3 The PISA 2012 index of economic, social, and cultural status (ESCS) was created using student reports on parental occupation, the highest level of 
parental education, and an index of home possessions related to family wealth, home educational resources and possessions related to “classical” 
culture in the family home. The home possessions relating to “classical” culture in the family home included possessions such as works of classical 
literature, books of poetry, and works of art (e.g., paintings). The cut points are calculated using the whole U.S. PISA 2012 sample. 

NOTE: ESO stands for Education and Skills Online. Results reported are weighted estimates for the PISA YAFS population, which is nationally 
representative of PISA 2012 students 4 years later at approximately age 19. Estimates for 19-year-olds are for individuals who were 15-year-old students 
in the fall of 2012 and who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. Detail may not sum 
to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-19. Percentage distribution of U.S. 19-year-olds and correlations between PISA 2012 reading scores and 
ESO literacy scores, by selected behavioral and affective characteristics: 2016 


Correlations between 
ed ke yAWAUM Ae =¥-Le [ale Mcxexe) «-1-) 
and ESO literacy scores in 


Total 2016 
Behavioral or affective characteristics 
Total 100.0 (tT) 0.59 (0.025) 
Frequency of skipping whole school day 
None 78.2 (1.59) 0.57 (0.027) 
One or more 21.8 (1.59) 0.63 (0.050) 
Openness to problem solving’ 
Bottom quarter 23.9 (1.47) 0.53 (0.081) 
Second quarter 24.4 (1.62) 0.52 (0.068) 
Third quarter 25.6 (1.84) 0.61 (0.039) 
Top quarter 26.1 (1.38) 0.61 (0.045) 
“! feel happy at school.” 
Strongly agree and agree 80.0 (1.63) 0.61 (0.039) 
Disagree and strongly disagree 20.0 (1.63) 0.52 (0.081) 


t Not applicable. 

‘The PISA 2012 openness to problem solving index was created using student responses to questions asking about the extent to which they feel they 
resemble someone who can handle a lot of information, is quick to understand things, seeks explanations for things, can easily link facts together and 
likes to solve complex problems. The cut points are calculated using the whole U.S. PISA 2012 sample. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-20. Percentage distribution of U.S. 19-year-olds and correlations between PISA 2012 mathematics scores and 
ESO numeracy scores, by selected behavioral and affective characteristics: 2016 
Correlations between 
PISA 2012 mathematics 
scores and ESO numeracy 


scores in 2016 


Behavioral or affective characteristics [Standard errors appear in parentheses] 


Total 100.0 (t) 0.69 (0.018) 

Frequency of skipping whole school day 

None 78.2 (1.57) 0.69 (0.024) 

One or more 21.8 (1.57) 0.68 (0.047) 
Openness to problem solving! 

Bottom quarter 23.7 (1.45) 0.60* (0.058) 

Second quarter 25.0 (1.55) 0.60* (0.053) 

Third quarter 25.3 (1.77) 0.70 (0.040) 

Top quarter 26.0 (1.38) 0.72 (0.029) 
“I feel happy at school.” 

Strongly agree and agree 79.8 (1.60) 0.72 (0.023) 

Disagree and strongly disagree 20.2 (1.60) 0.67 (0.053) 


t Not applicable. 

* The correlation statistic for the subgroup is significantly different from the correlation statistic for the total (p < .05). 

’ The PISA 2012 openness to problem solving index was created using student responses to questions asking about the extent to which they feel they 
resemble someone who can handle a lot of information, is quick to understand things, seeks explanations for things, can easily link facts together and 
likes to solve complex problems. The cut points are calculated using the whole U.S. PISA 2012 sample. 

NOTE: ESO stands for Education and Skills Online. Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and 
who participated in PISA YAFS in 2016. As part of PISA YAFS, participants completed the ESO assessments. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-21. Percentage distribution of U.S. 19-year-olds, by PISA 2012 reading proficiency levels and selected 
education characteristics: 2016 


PISA 2012 reading proficiency levels 


Low Middle High (Level 5 
ate lUfer-1itola ed afelestei stato ttecs Total (Below level 2) (Levels 2—4) Vale i-leley'(:)) 


in 2016 [Standard errors appear in parentheses] 
Total 100.0 (t) 100.0 (t) 100.0 (t) 100.0 (t) 
Degree currently pursued 
Bachelor’s degree and above 45.2 (1.98) 12.4* (3.26) 46.6 (2.03) 83.9* (3.52) 
Pre-associate’s or associate’s 
degree 20.1 (1.29) 15.3 (4.22) 22.1 (1.47) 10.8* (2.77) 
High school diploma 8.8 (1.12) 23.3* (5.65) 7.1. (1.10) t (t) 
Not currently studying, has a 
high school diploma 19.5 (1.53) 31.7 (5.73) 19.2 (1.65) t (t) 
Not currently studying, does not 
have a high school diploma 6.3 (1.20) 17.4! (5.90) 5.1. (1.22) t (tT) 
Area of study currently pursued 
Social sciences 15.0 (1.03) t (tT) 16.5 (1.25) 17.9 (3.03) 
Health 12.8 (0.85) 5.3! (2.04) 14.1. (1.03) 12.7 (2.65) 
Sciences 11.6 (0.81) t (t) 11.6 (1.02) 26.6* (3.61) 
Engineering 7.1 (0.66) 3.2 ! (1.38) 6.8 (0.69) 15.2* = (2.72) 
Humanities 6.1 (0.60) t (t) 6.0 (0.75) 13.9* (2.75) 
General programs 4.8 (0.62) t (t) 5.1 (0.70) t (t) 
Teacher training 4.6 (0.53) t (t) 5.2 (0.64) 4.5! (2.08) 
Services 2.4 (0.36) 3.8 ! (1.53) 2.2 (0.41) t (tT) 
Agriculture 1.0 (0.24) # (t) 1.1 (0.30) t (t) 
Currently pursuing a high school 
diploma 8.8 (1.12) 23.3* (5.65) 7.1. (1.10) t (tT) 
Not currently studying 25.9 (1.55) 49.1* (6.37) 24.3 (1.72) 3.8!* (1.85) 
Nonformal education 
participation in the last 
12 months’ 
Participated in nonformal 
education 66.4 (1.45) 58.7 (5.64) 67.5 (1.63) 69.2 (3.73) 
Did not participate 33.2 (1.45) 40.5 (5.65) 32.3 (1.63) 30.4 (3.73) 
Missing 0.3! (0.13) t (t) t (t) t (t) 


t Not applicable. 

# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

$ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents in proficiency level category is significantly different from percentage of total U.S. PISA YAFS population for the given 
education characteristic (p < .05). 

' Nonformal education includes distance education, job training, seminars, and private training. 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. 
The education characteristics data in this table are from a background questionnaire that participants completed as a part of PISA YAFS. See exhibit 2 
for descriptions of proficiency levels. Data for “Area of study currently pursued” excludes five missing cases. Percentages may thus differ slightly from 
their analogous categories in the “Degree currently pursued” variable, for which there were no missing cases. Detail may not sum to totals because of 
rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-22. — Percentage distribution of U.S. 19-year-olds, by PISA 2012 mathematics proficiency levels and selected 


education characteristics: 2016 
PISA 2012 mathematics proficiency levels 


Low Middle High (Level 5 
Education characteristics Total (Below level 2) (Levels 2—4) Vale ir-1eXe'(:)) 


in 2016 [Standard errors appear in parentheses] 
Total 100.0 (t) 100.0 (t) 100.0 (t) 100.0 (t) 
Degree currently pursued 
Bachelor’s degree and above 45.1 (2.00) 14.8* (2.35) 49.0 (2.15) 86.3* (2.63) 
Pre-associate’s or associate’s 
degree 20.0 (1.29) 18.5 (3.15) 22.1 (1.59) 8.5* = (2.20) 
High school diploma 9.0 (1.10) 22.2* (3.73) 5.7 (1.05) t (t) 
Not currently studying, has a 
high school diploma 19.7 (1.50) 30.4 (4.08) 18.7 (1.81) 3.4!* (1.59) 
Not currently studying, does not 
have a high school diploma 6.2 (1.19) 14.1 (4.18) 4.5 (1.07) t (tT) 
Area of study currently pursued 
Social sciences 14.9 (1.00) 6.3!* (2.36) 17.3 (1.36) 17.8 (2.89) 
Health 12.5 (0.83) 8.4 (1.66) 14.3 (1.10) 9.0 (2.24) 
Sciences 11.7 (0.83) 3.5!* (1.23) 11.9 (1.07) 29.1* (4.16) 
Engineering 7.1 (0.65) 2.3!* (1.06) 6.6 (0.90) 20.9* (3.40) 
Humanities 6.2 (0.61) 2.7! (1.27) 6.5 (0.80) 11.8 (2.68) 
General programs 4.9 (0.58) 4.9! (1.75) 5.4 (0.68) t (t) 
Teacher training 4.5 (0.49) 2.7! (1.29) 5.3 (0.72) 3.7! (1.51) 
Services 2.3 (0.35) 2.4! (1.08) 2.4 (0.50) t (t) 
Agriculture 1.0 (0.24) # (t) 1.4 (0.33) t (t) 
Currently pursuing a high school 
diploma 9.0 (1.10) 22.2* (3.74) 5.7 (1.05) t (t) 
Not currently studying 26.0 (1.58) 44.6* (4.55) 23.2 (1.76) 3.6!* (1.54) 


Nonformal education 
participation in the last 


12 months’ 
Participated in nonformal 

education 66.7 (1.44) 64.0 (4.14) 67.7 (1.70) 66.1 (3.82) 
Did not participate 32.9 (1.43) 35.3 (4.16) 32.1. (1.70) 33.1 (3.95) 
Missing 0.4! (0.14) t (T) - (T) tf (T) 


t Not applicable. 

# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents in proficiency level category is significantly different from percentage of total U.S. PISA YAFS population for the given 
education characteristic (p < .05). 

‘ Nonformal education includes distance education, job training, seminars, and private training. 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. 
The education characteristics data in this table are from a background questionnaire that participants completed as a part of PISA YAFS. See exhibit 3 
for descriptions of proficiency levels. Data for “Area of study currently pursued” excludes two missing cases. Percentages may thus differ slightly from 
their analogous categories in the “Degree currently pursued” variable, for which there were no missing cases. Detail may not sum to totals because of 
rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-23. 


Percentage distribution of U.S. 19-year-olds, by PISA 2012 reading proficiency levels and selected 
employment, education, and occupation status: 2016 


PISA 2012 reading proficiency levels 
Total 


High (Level 5 
and above) 


Low Middle 
(Below level 2) (URW Ee) 


[Standard errors appear in parentheses] 


tia} e) Coy antclaemevele(er-1iela mer: lare| 
occupation status in 2016 


Total 100.0 (t) 100.0 (t) 100.0 (t) 100.0 (t) 
Employment status 
Unemployed (not looking for 
work) 24.4 (1.36) 18.3 (5.45) 23.0 (1.65) 45.9* (4.11) 
Unemployed (looking for work) 18.3 (1.22) 25.9 (4.85) 18.2 (1.44) 8.1!* (2.52) 
Apprentice/intern 1.2 (0.24) t (t) 1.2 (0.32) t (t) 
Part-time employed 39.9 (1.70) 31.1 (5.74) 41.7 (2.01) 37.4 (3.78) 
Full-time employed 13.4 (1.05) 17.3 (4.01) 13.6 (1.12) 5.1! = (1.88) 
Other 2.8 (0.83) t (t) 2.3! (0.98) t (t) 
Combined employment and 
education status 
In employment only! 16.2 (1.18) 29.2 (5.67) 15.5 (1.38) t (t) 
In formal education only 34.9 (1.73) 31.7 (6.78) 33.5 (2.14) 52.4* (4.84) 
Both in employment and 
formal education 38.3 (1.54) 19.2* (4.36) 41.1 (1.69) 43.2 (4.31) 
Neither employed nor in 
formal education 7.8 (1.19) 12.5! (4.04) 7.6 (1.26) t (t) 
In formal education, employment 
status other? 0.9 (0.26) # (t) 1.1 (0.33) t (t) 
Not in formal education, 
employment status other? 1.9! (0.75) t (t) t (t) t (t) 
Current occupation® 
Armed forces occupations 2.0 (0.44) t (t) 2.3 (0.61) t (t) 
Managers 2.7 (0.43) t (t) 3.2 (0.51) t (t) 
Professionals 4.6 (0.62) t (t) 4.6 (0.64) 7.0 (2.07) 
Technicians and associate 
professionals 4.0 (0.55) t (t) 4.1 (0.61) 4.3! (1.66) 
Clerical support workers 2.9 (0.44) t (t) 3.1 (0.49) 2.1! (0.97) 
Service and sales workers 30.7 (1.47) 32.9 (6.06) 30.8 (1.63) 26.3 (3.92) 
Skilled agricultural, forestry and 
fish 0.9 (0.26) t (t) 1.0! (0.30) t (t) 
Craft and related trades workers 4.4 (0.62) 6.1! (2.44) 4.5 (0.78) t (t) 
Plant and machine operators, 
and assembly 2.0 (0.48) t (t) 1.8! (0.56) t (t) 
Elementary occupations* 3.1 (0.49) t (tT) 3.5 (0.61) t (tT) 


See notes at end of table. 
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Table A-23. = Percentage distribution of U.S. 19-year-olds, by PISA 2012 reading proficiency levels and selected 
employment, education, and occupation status: 2016—Continued 


PISA 2012 reading proficiency levels 
Total 


Low Middle High (Level 5 
(Below level 2) (Levels 2—4) and above) 


Employment, education, and 


occupation status in 2016 [Standard errors appear in parentheses] 
Unemployed (not looking for 
work) 24.4 (1.36) 18.3 (5.45) 23.0 (1.65) 45.9* (4.11) 
Unemployed (looking for work) 18.3 (1.22) 25.9 (4.85) 18.2 (1.44) 8.1! (2.52) 


t Not applicable. 

# Rounds to zero. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents in proficiency level category is significantly different from percentage of total U.S. PISA YAFS population for the given 
employment and education characteristic (p < .05). 

1 Employment is defined as being either full- or part-time employed or an apprentice/intern. 

? Employment status other indicates “other” was selected. 

3 Current occupation is for those employed full or part time or as an apprentice/intern. 

4 Elementary occupations include crop farm laborer, shelf filler, kitchen helper, refuse worker, etc. 

NOTE: Results reported are weighted estimates for the PISA YAFS population, which is nationally representative of PISA 2012 students 4 years later 
at approximately age 19. Data on employment, education, and occupation status in this table are from a background questionnaire that participants 
completed as a part of PISA YAFS. See exhibit 2 for descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 


Table A-24. —_ Percentage distribution of U.S. 19-year-olds, by PISA 2012 mathematics proficiency levels and selected 
employment, education, and occupation status: 2016 


PISA 2012 mathematics proficiency levels 


Low Middle High (Level 5 
Employment, education, and a Ke) r-1| (Below level 2) (Levels 2—4) lace m-lexey:)) 


occupation status in 2016 [Standard errors appear in parentheses] 
Total 100.0 (t) 100.0 (t) 100.0 (t) 100.0 (t) 
Employment status 
Unemployed (not looking for 


work) 24.3 (1.35) 16.5 (3.73) 23.5 (1.75) 46.4* (3.32) 
Unemployed (looking for work) 18.5 (1.20) 25.7 (3.47) 17.5 (1.46) 8.9* (1.95) 
Apprentice/intern 1.3 (0.25) t (t) 1.3 (0.37) 3.1! (1.40) 
Part-time employed 39.7 (1.69) 35.1 (3.74) 42.0 (1.96) 34.6 (3.78) 
Full-time employed 13.6 (1.06) 15.9 (2.84) 13.9 (1.35) 5.8 !* (1.75) 
Other 2.8 (0.82) 6.4! (2.87) 1.8! (0.63) t (t) 


Combined employment and 
education status 


In employment only! 16.3 (1.22) 27.7* (3.81) 14.7 (1.42) t (t) 
In formal education only 34.9 (1.70) 31.0 (4.17) 33.4 (1.92) 53.9* (3.59) 
Both in employment and 

formal education 38.2 (1.53) 23.6* (3.31) 42.5 (1.87) 41.7 (3.83) 
Neither employed nor in 

formal education 7.8 (1.15) 11.2 (2.68) 7.7 (1.34) t (t) 
In formal education, employment 

status other? 0.9 (0.26) t (t) 0.9! (0.32) t (t) 
Not in formal education, 

employment status other? 1.9! (0.74) 5.6! (2.78) t (t) t (t) 


See notes at end of table. 
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Table A-24. Percentage distribution of U.S. 19-year-olds, by PISA 2012 mathematics proficiency levels and selected 
employment, education, and occupation status: 2016—Continued 


PISA 2012 mathematics proficiency levels 


Low Middle High (Level 5 
Employment, education, and Total (Below level 2) (Levels 2—4) Vale ir-leley'(:)) 


occupation status in 2016 [Standard errors appear in parentheses] 
Current occupation? 
Armed forces occupations 1.9 (0.44) t (t) 2.3 (0.65) t (t) 
Managers 2.7 (0.42) t (t) 3.4 (0.61) t (t) 
Professionals 4.6 (0.62) 3.7! (1.56) 4.4 (0.67) 7.8 (1.72) 
Technicians and associate 
professionals 4.0 (0.55) 3.9! (1.57) 4.1 (0.69) 3.6 ! (1.29) 
Clerical support workers 2.9 (0.42) 2.6! (1.12) 3.0 (0.51) 2.7 | (1.16) 
Service and sales workers 30.6 (1.44) 33.4 (4.34) 30.9 (1.66) 22.9 (3.43) 
Skilled agricultural, forestry and 
fish 1.0 (0.26) t (t) 1.0! (0.33) t (t) 
Craft and related trades workers 4.4 (0.62) 4.8 (1.13) 4.6 (0.83) t (Tt) 
Plant and machine operators, 
and assembly 2.0 (0.48) 2.9! (1.27) 1.9 (0.54) t (tT) 
Unemployed (not looking for 
work) 24.3 (1.35) 16.5 (3.73) 23.5 (1.75) 46.4* (3.32) 
Unemployed (looking for work) 18.5 (1.20) 25.7 (3.47) 17.5 (1.46) 8.9* (1.95) 


t Not applicable. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

+ Reporting standards not met. The coefficient of variation (CV) for this estimate is 50 percent or greater. 

* Percentage of respondents in proficiency level category is significantly different from percentage of total U.S. PISA YAFS population for the given 
employment and education characteristic (p < .05). 

' Employment is defined as being either full- or part-time employed or an apprentice/intern. 

2 Employment status other indicates “other” was selected. 

3 Current occupation is for those employed full or part time or as an apprentice/intern. 

4 Elementary occupations include crop farm laborer, shelf filler, kitchen helper, refuse worker, etc. 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. Data 
on employment, education, and occupation status in this table are from a background questionnaire that participants completed as a part of this study in 
2016. See exhibit 3 for descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-25. Percentage distribution of U.S. 19-year-olds’ personality characteristics and their averages on the 
vocational interests index, by PISA 2012 reading proficiency levels: 2016 


PISA 2012 reading proficiency levels 
Total 


oN Middle High (Level 5 
(Below level 2) (Levels 2—4) and above) 


Personality characteristics and 
EW(-1¢-(e(-\- Mola aal- Wa doler-1t(olarel 


interests index in 2016 [Standard errors appear in parentheses] 
Total 100.0 (t) 100.0 (Tt) 100.0 (tT) 100.0 (Tt) 
Level of self-efficacy’ 
Low 7.9 (0.97) 14.2! (4.78) 74 (1.02) 3.2! (1.45) 
Moderate 39.5 (1.44) 37.4 (5.81) 40.8 (1.66) 30.9 (3.81) 
High 21.8 (1.27) 20.8 (5.45) 22.7 (1.48) 16.2 (3.03) 
Not applicable 24.4 (1.36) 18.3 (5.45) 23.0 (1.65) 45.9* (4.11) 
Missing 6.4 (0.70) 9.3! (2.89) 6.2 (0.80) 3.7! (1.29) 
Life satisfaction 
Low 20.9 (1.24) 23.5 (5.38) 21.1 (1.31) 15.0 (2.68) 
Moderate 36.9 (1.20) 30.3 (6.45) 37.9 (1.75) 38.4 (4.14) 
High 33.6 (1.36) 31.1 (5.52) 33.2 (1.65) 40.2 (4.43) 
Missing 8.6 (0.84) 15.1 (4.40) 7.7 (0.83) 6.3 (1.69) 


Average on the vocational 
interests index? 


Realistic 15.1 (0.31) 16.8 (1.21) 15.0 (0.33) 13.8 (0.82) 
Investigative 17.6 (0.28) 16.0 (1.15) 17.5  (0.36)| 20.5* (0.81) 
Artistic 17.7. (0.31) 16.0 (1.38) 17.8 (0.39) 18.9 (0.91) 
Social 20.8 (0.25) 19.7 (0.94) 21.1 (0.33) 20.0 (0.69) 
Enterprising 19.4 (0.25) 19.6 (1.52) 19.6 (0.31) 17.4* (0.61) 
Conventional 15.4 (0.30) 15.9 (1.45) 15.4 (0.36) 14.6 (0.64) 


t Not applicable. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

* Percentage of respondents in proficiency level category is significantly different from percentage of total U.S. PISA YAFS population for the given 
personality characteristic (p < .05). 

' Self-efficacy questions were not asked from the unemployed, not looking for work (out of the labor force) 19-year-olds and these data are marked as 
“Not applicable” in this table. 

? The scores on the vocational scales range from 0 to 40. The realistic type of individual is interested in manual types of jobs, the investigative type 

is Curious and scientifically oriented, the artistic type prefers creative activities through different art forms, the social type is interested in working with 
people, the enterprising type leads and influences people, and the conventional type prefers well-structured situations. For more information on the 
vocational scales and interest categories, see OECD 2015. 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. The 
personality characteristics data in this table are from a background questionnaire that participants completed as a part of PISA YAFS. See exhibit 2 for 
descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Table A-26. Percentage distribution of U.S. 19-year-olds’ personality characteristics and their averages on the 
vocational interests index, by PISA 2012 mathematics proficiency levels: 2016 


PISA 2012 mathematics proficiency levels 


Personality characteristics and Low Middle High (Level 5 
averages on the vocational Total (Below level 2) ((R=\(-) 9) lace m-leley(:)) 


interests index in 2016 [Standard errors appear in parentheses] 
Total 100.0 (t) 100.0 (t) 100.0 (t) 100.0 (t) 
Level of self-efficacy’ 
Low 7.8 (0.95) 11.7 (2.84) 7.1 (1.12) 3.9! (1.66) 
Moderate 38.8 (1.45) 40.2 (4.42) 39.7 (1.89) 29.7 (3.82) 
High 21.5 (1.26) 21.8 (4.23) 22.3 (1.44) 15.6 (3.04) 
Not applicable 24.3 (1.35) 16.5 (3.73) 23.5 (1.75) 46.4* (3.32) 
Missing 7.6 (0.64) 9.7 (2.33) 7.4 (0.73) 44! (1.77) 
Life satisfaction 
Low 20.6 (1.21) 23.6 (3.39) 20.6 (1.32) 13.9 (2.63) 
Moderate 36.3 (1.24) 35.7 (4.54) 36.1. (1.61) 39.7 (3.36) 
High 33.0 (1.31) 28.0 (3.58) 33.7 (1.53) 39.6 (3.17) 
Missing 10.0 (0.84) 12.6 (3.15) 9.6 (0.83) 6.9! (2.37) 


Average on the vocational 
interests index? 


Realistic 15.1 (0.31) 15.3. (0.78) 15.1 (0.37) 15.0 (0.88) 
Investigative 17.6 (0.28) 15.7* (0.69) 17.8 (0.37) 20.4* (0.84) 
Artistic 17.7. (0.31) 16.9 (0.73) 17.8 (0.37) 18.6 (0.81) 
Social 20.8 (0.25) 20.1 (0.61) 21.2 (0.33) 19.5 (0.57) 
Enterprising 19.4 (0.25) 20.0 (1.01) 19.5 (0.29) 17.7* (0.43) 
Conventional 15.4 (0.30) 15.7 (0.84) 15.3 (0.32) 15.1 (0.68) 


t Not applicable. 

! Interpret data with caution. The coefficient of variation (CV) for this estimate is between 30 and 50 percent. 

* Percentage of respondents in proficiency level category is significantly different from percentage of total U.S. PISA YAFS population for the given 
personality characteristic (p < .05). 

' Self-efficacy questions were not asked from the unemployed, not looking for work (out of the labor force) 19-year-olds and these data are marked as 
“Not applicable’ in this table. 

? The scores on the vocational scales range from 0 to 40. The realistic type of individual is interested in manual types of jobs, the investigative type 

is curious and scientifically oriented, the artistic type prefers creative activities through different art forms, the social type is interested in working with 
people, the enterprising type leads and influences people, and the conventional type prefers well-structured situations. For more information on the 
vocational scales and interest categories, see OECD 2015. 

NOTE: Estimates for 19-year-olds are for individuals who were 15-year-old students in the fall of 2012 and who participated in PISA YAFS in 2016. The 
personality characteristics data in this table are from a background questionnaire that participants completed as a part of PISA YAFS. See exhibit 3 for 
descriptions of proficiency levels. Detail may not sum to totals because of rounding. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
Program for International Student Assessment Young Adult Follow-up Study (PISA YAFS), 2016. 
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Appendix B. 
Glossary of Terms 


The variables used in the analyses for this report are 
described below. The names of variables that were used 
to produce estimates for this report appear in capital 
letters. Since the variables come from the PISA YAFS 
public-use data file (NCES 2021, forthcoming) as well 

as from the PISA 2012 public-use data file (NCES 2014), 
the source data file is noted in parentheses next to the 
variables. In some cases, the variables have been used in 
the exact format in which they appear in the data file. In 
other cases, variables available in the data file have been 
modified (e.g., when the categories have been combined 
to create a smaller number of categories). Such collapsing 
of categories and the names of original variables are 
noted in the descriptions. In other cases, new measures 
have been created specifically for this report by 
combining information from two or more variables in 
the data file. In these instances, the names of original 
variables used to create the new measure are also noted. 


In this analysis, 3 percent or less of PISA 2012 variables’ 
data for gender; race/ethnicity; nativity; language spoken 
at home; parents’ education; school location; school 
type; percentage of students receiving free or reduced- 
price lunch; index of economic, social, and cultural 
status; and frequency of skipping a whole school day 

are missing. Index of openness to problem solving and a 
variable of “I feel happy at school” have higher rates of 
missing (35 and 37 percent, respectively). Cases that are 
missing are excluded for the analysis of that variable. 


Variables 


PISA 2012 reading proficiency levels (PISA 2012): 
These variables are derived based on the PISA 2012 
reading plausible values, PVIREAD, PV2READ, PV3READ, 
PV4READ, and PV5READ. The average of the five 
plausible values is used in the derivation of proficiency 
levels. If the respondent’s average reading plausible 
values are between O (included) and 407.47 (included), 
the proficiency level is defined as below level 2. If the 
respondent’s average reading plausible values are 
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between 407.47 (not included) and 625.61 (included), 

the proficiency level is defined as level 2 to level 4. If 

the respondent’s reading average plausible values are 
between 625.61 (not included) and 1,000 (included), the 
proficiency level is defined as level 5 and above. (Note 
that these score-point thresholds have been rounded in 
exhibit 2.) For more details on the estimation procedures 
using plausible values, refer to Chapter 8 of the PISA 
YAFS Technical Report (Kastberg et al. forthcoming). 


The values for PISA 2012 reading proficiency levels are as 
follows: 

1 = Below level 2 

2 = Level 2 to level 4 

3 = Level 5 and above 


PISA 2012 mathematics proficiency levels (PISA 
2012): These variables are derived based on PISA 2012 
mathematics plausible values, PVIMATH, PV2MATH, 
PV3MATH, PV4MATH, and PV5MATH. The average 

of the five plausible values is used in the derivation 

of proficiency levels. If the respondent’s average 
mathematics plausible values are between O (included) 
and 420.07 (included), the proficiency level is defined as 
below level 2. If the respondent’s average mathematics 
plausible values are between 420.07 (not included) and 
606.99 (included), the proficiency level is defined as 
level 2 to level 4. If the respondent’s average mathematics 
plausible values are between 606.99 (not included) and 
1000 (included), the proficiency level is defined as level 
5 and above. (Note that these score-point thresholds 
have been rounded in exhibit 3.) For more details on the 
estimation procedures using plausible values, refer to 
Chapter 8 of the PISA YAFS Technical Report (Kastberg et 
al. forthcoming). 


The values for PISA 2012 mathematics proficiency levels 
are as follows: 

1 = Below level 2 

2 = Level 2 to level 4 

3 = Level 5 and above 


ESO literacy proficiency levels (PISA YAFS): These 
variables are derived based on the ESO literacy plausible 
values, PVLIT1-PVLIT10. In the analysis where only 
ESO literacy proficiency scores or levels are involved, 
an average of all 10 plausible values is used in the 
derivation of proficiency levels. When both PISA 
reading and ESO literacy proficiency scores or levels 
are involved, an average of the first five plausible values 
of the ESO are used in the derivation of proficiency 
levels. If the respondent’s average literacy plausible 
values are between O (included) and 226 (not included), 
the proficiency level is defined as below level 2. If 

the respondent’s average literacy plausible values 

are between 226 (included) and 326 (not included), 

the proficiency level is defined as level 2 to level 3. 

If the respondent’s average literacy plausible values 

are between 326 (included) and 500 (included), the 
proficiency level is defined as level 4 and above. For more 
details on the estimation procedures using plausible 
values, refer to Chapter 8 of the PISA YAFS Technical 
Report (Kastberg et al. forthcoming). 


The values for ESO literacy proficiency levels are as 
follows: 

1 = Below level 2 (ESO) 

2 = Level 2 to level 3 (ESO) 

3 = Level 4 and above (ESO) 


ESO numeracy proficiency levels (PISA YAFS): 
These variables are derived based on the ESO numeracy 
plausible values, PYNUM1-PVNUMIO. In the analysis 
where only ESO numeracy proficiency scores or levels 
are involved, an average of all 10 plausible values is 

used in the derivation of proficiency levels. When both 
PISA mathematics and ESO numeracy proficiency 
scores or levels are involved, an average of the first five 
plausible values of the ESO are used in the derivation of 
proficiency levels. If the respondent’s average numeracy 
plausible values are between O (included) and 226 (not 
included), the proficiency level is defined as below level 
2. If the respondent’s average numeracy plausible values 
are between 226 (included) and 326 (not included), 

the proficiency level is defined as level 2 to level 3. If 
the respondent’s average numeracy plausible values 

are between 326 (included) and 500 (included), the 
proficiency level is defined as level 4 and above. For more 
details on the estimation procedures using plausible 
values, refer to Chapter 8 of the PISA YAFS Technical 
Report (Kastberg et al. forthcoming). 


The values for ESO numeracy proficiency levels are as 
follows: 
1 = Below level 2 (ESO) 
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2 = Level 2 to level 3 (ESO) 
3 = Level 4 and above (ESO) 


Gender (PISA 2012): The variable indicates the 
respondent’s sex. It comes directly from the respondent’s 
response to the survey question about their sex, 
STO4Q01. 


The values for gender are as follows: 
1= Male 
2 = Female 


Race/Ethnicity (PISA 2012): The variable indicates the 
respondent’s race and ethnicity. The variable is derived 
from the respondent’s response to the survey question 
about their race and ethnicity, RACETHC. The values of 
this variable are the same with the values of RACETHC, 
with only one exception: When RACETHC is multiracial 
or other, the race/ethnicity variable is coded as other. 


The values for race/ethnicity are as follows: 
1 = White 

2 = Black or African American 

3 = Hispanic or Latino 

4 = Asian 

5 = Other 


Nativity (PISA 2012): The variable is derived from 

the respondent’s country of birth (ST20Q01), the 
respondent’s parents’ country of birth (ST20Q02 and 
ST20Q03), and the respondent’s family structure 
(FAMSTRUC). If the respondent and one or both parents 
were born in the United States, the variable is coded as 
native born, parent native. If the respondent was born in 
the United States and both parents were born outside the 
United States or the respondent was born in the United 
States and lives in a single-parent family with the parent 
born outside the Unites States, the variable is coded as 
native born, parent(s) nonnative. If the respondent was 
not born in the United States, regardless of the nativity 
status of the parent, the variable is coded as nonnative. 


The values for nativity are as follows: 
1 = Native born, parent native 

2 = Native born, parent(s) nonnative 
3 = Nonnative 


Language spoken at home (PISA 2012): The variable 
indicates the language the respondent speaks at home. It 
is a derived variable based on the respondent’s response 
to the survey question about the language they speak at 
home, LANGN. 


The values for language spoken at home are as follows: 
1= English 

2 = Spanish 

3 = Other languages 


Parents’ Education (PISA 2012): The variable 

indicates the highest level of education completed by the 
respondent’s parents. It is a derived variable based on the 
respondent’s parents’ education set into the International 
Standard Classification of Education (ISCED) categories, 
HISCED. When the parents’ ISCED is none, 1, 2, 3C, 3B, 
3A, or 4, the variable is coded as high school and below. 
When the parents’ ISCED is 5B, the variable is coded as 
associate’s degree. When the parents’ ISCED is 5A or 6, 
the variable is coded as bachelor’s degree or higher. 


The values for parents’ education are as follows: 
1 = High school and below 

2 = Associate’s degree 

3 = Bachelor’s degree or higher 


School Location (PISA 2012): The variable indicates the 
types of the school location. It comes directly from the 
response of the respondent’s school administrator to the 
survey question about the school location, SCO3Q01. 


The values for school location are as follows: 
1= Village 

2 = Small town 

3 = Town 

4 = City 

5 = Large city 


School Type (PISA 2012): The variable indicates 
whether the school is a public or private school. It 

is a derived variable based on the response of the 
respondent’s school administrator to the survey question 
about the school type, PUBPRIV. 


The values for school type are as follows: 
1 = Public 
2 = Private 


Percentage of Free or Reduced-Price Lunch (PISA 
2012): The variable indicates the percentage of students 
enrolled in the free or reduced-price lunch program in 
the school. It is a derived variable based on the response 
of the respondent’s school administrator to the survey 
question about the percentage of students enrolled in the 
free or reduced-price lunch program in the respondent’s 
school, FRPL. 
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The values for percentage of free or reduced-priced lunch 
are as follows: 

1 = Less than 25 percent 

2 = 25 to 49.9 percent 

3 = 50 to 74.9 percent 

4 = 75 percent or more 


Index of economic, social, and cultural status (PISA 
2012): This variable is derived from several variables 
such as parental education and occupation, the number 
and type of home possessions that are considered 
proxies for wealth, and the number of books and other 
educational resources available at home. For more 

on the index, refer to PISA 2012 Results (Volume II), 

Box II.2.1 (OECD 2013c). Students are categorized into 
quarters based on the index of economic, social, and 
cultural status. To assign students with the same index 
into different quarters, a random subsampling process is 
implemented by adding a small random variable to the 
index before dividing the index variable into quarters. 
The procedure is documented in the PISA Data Analysis 
Manual (2nd ed.), Chapter 14 (OECD 2009). 


The values for index of economic, social, and cultural 
status are as follows: 

1= Bottom quarter 

2 = Second quarter 

3 = Third quarter 

4 = Top quarter 


Frequency of skipping whole school day (PISA 
2012): The variable indicates the number of times 

the respondent skipped a whole school day in the last 
two full weeks of school. It comes directly from the 
respondent’s response to the survey question about the 
number of times in the last two full weeks of school they 
skipped a whole school day, STO9Q01. When STO9Q01 
is none, the variable is coded as none. When STO9QO1 
is one or two times, three or four times, or five or more 
times, the variable is coded as one or more. 


The values for frequency of skipping whole school day 
are as follows: 

1= None 

2 = One or more 


Index of openness to problem solving (PISA 2012): 
The variable is based on students’ responses to questions 
asking about the extent to which they feel they resemble 
someone who can handle a lot of information, is quick 
to understand things, seeks explanations for things, 

can easily link facts together, and likes to solve complex 


problems. Students are categorized into four equally 
sized quarters based on the index of openness to 
problem solving. To assign students with the same index 
into different quarters, a random subsampling process is 
implemented by adding a small random variable to the 
index before dividing the index variable into quarters. 
The procedure is documented in the PISA Data Analysis 
Manual (2nd ed.), Chapter 14 (OECD 2009). 


The values for index of openness to problem solving are 
as follows: 

1 = Bottom quarter 

2 = Second quarter 

3 = Third quarter 

4 = Top quarter 


I feel happy at school (PISA 2012): The variable 
indicates whether the respondent feels happy at school. 
It comes directly from the respondent’s response to the 
survey question on the extent to which they agree with 
the “I feel happy at school” statement, ST87QO7. 


The values for I feel happy at school are as follows: 
1 = Strongly agree or agree 
2 = Disagree or strongly disagree 


Degree currently pursued (PISA YAFS): The variable 
indicates the degree the respondent is currently 
pursuing. It is derived from the respondent’s response to 
the survey question on what type of degree or certificate 
they are currently studying for, B QO2BUSYAFS, and their 
response to the survey question on the highest education 
level they have completed, BQ _Q3. When B_QO2BUSYAFS 
is high school diploma, the variable is coded as high 
school diploma. When B_ QO2BUSYAFS is pre-associate/ 
trade without a degree, associate/trade certificate, 

or associate’s degree, the variable is coded as pre- 
associate’s or associate’s degree. When B QO2BUSYAFS is 
bachelor’s degree, master’s degree, professional degree, 
or doctorate degree, the variable is coded as bachelor’s 
degree or above. When B QO2BUSYAFS is missing and 

BQ _Q3 is secondary education, some postsecondary 
education, 4-year college or university degree, or beyond 
a college or university degree, the variable is coded as 
not currently studying and has a high school diploma. 
When B QO2BUSYAFS is missing and BQ Q3 is no 
education, primary education, or secondary education 
without a diploma, the variable is coded as not currently 
studying and doesn’t have a high school diploma. When 
B_QO2BUSYAFS is missing and BQ Q3 is missing, the 
variable is coded as not currently studying and the degree 
is unknown. 
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The values for degree currently pursued are as follows: 

1 = High school diploma 

2 = Pre-associate’s or associate’s degree 

3 = Bachelor’s degree or above 

4 = Not currently studying and has a high school diploma 
5 = Not currently studying and doesn’t have a high school 
diploma 

6 = Not currently studying and the degree is unknown 


Area of study currently pursued (PISA YAFS): The 
variable indicates the area of study the respondent is 
currently pursuing. It is derived from the respondent’s 
response to the survey question on what type of 

degree or certificate they are currently studying for, 
B_QO2BUSYAFS, and their response to the follow-up 
survey question on the area of study, emphasis, or major 
for the degree or certificate they are currently studying 
for (if more than one, they were asked to choose the one 
they considered most important), B_ QO2C. Most of the 
values come directly from B_QO2C. Other than that, when 
B_QO2C is NA, the variable is coded as currently pursuing 
a high school diploma. When B_QO2BUSYAFS is missing, 
the variable is coded as not currently studying. 


The values for area of study currently pursued are as 
follows: 

1 = General programs 

2 = Teacher training 

3 = Humanities/languages/arts 

4 = Social sciences/business/law 

5 = Sciences/math/computing 

6 = Engineering 

7 = Agriculture/veterinary 

8 = Health/welfare 

9 = Services 

997 = Currently pursuing a high school diploma 
998 = Not currently studying 


Participation in nonformal education (PISA 

YAFS): The variable indicates whether the respondent 
participated in any nonformal education in the last 12 
months, including distance education, job training, 
seminars, or private training. The variable is derived 
from the respondent’s response to the survey questions 
on organized learning activities they participated in 
during the last 12 months, including both work and 
nonwork-related activities: B Q12A (distance education), 
B_Q12C (job training), B_Q1I2E (seminars), and B_Q12G 
(private training). If the respondent attended any of 
these four activities, the variable is coded as participated 
in nonformal education. If the respondent attended none 


of the above four activities and none of the four activities 
is missing, the variable is coded as not participated in 
nonformal education. If the respondent attended none of 
the above four activities and any of the four activities are 
missing, the variable is missing. 


The values for participation in nonformal education are 
as follows: 

1 = Participated in nonformal education 

2 = Not participated in nonformal education 

999 = Missing 


Employment status (PISA YAFS): The variable indicates 
the respondent’s current employment status. It comes 
directly from the respondent’s response to the survey 
question on which of the following best describes their 
current employment status, BQ Q8. 


The values for employment status are as follows: 
1 = Unemployed (not looking for work) 

2 = Unemployed (looking for work) 

3 = Apprentice/Intern 

4 = Part-time employed 

5 = Full-time employed 

9 = Other 

999 = Missing 


Combined employment and education status (PISA 
YAFS): This variable is derived from the respondent’s 
current employment status, BQ Q8, and respondent’s 
current studying status, B_QO2A. If the respondent is 
not currently studying and employed (full-time/part- 
time or employed as an intern), the variable is coded 

as in employment only. If the respondent is currently 
studying and is unemployed (not looking for work/ 
looking for work), the variable is coded as in formal 
education only. If the respondent is currently studying 
and employed (full-time/part-time or employed as an 
intern), the variable is coded as both in employment and 
in formal education. If the respondent is not currently 
studying and not employed (not looking for work/ 
looking for work), the variable is coded as in neither 
employment nor formal education. If the respondent is 
currently studying and the employment status is missing 
or other, the variable is coded as in formal education, 
employment status unknown. If the respondent is not 
currently studying and the employment status is missing 
or other, the variable is coded as not in formal education, 
employment status unknown. There is one edit to the 
data on file. 
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The values for Combined employment and education 
status are as follows: 

1= In employment only 

2 = In formal education only 

3 = Both in employment and in formal education 

4 = In neither employment nor formal education 

5 = In formal education, employment status unknown 

6 = Not in formal education, employment status unknown 


Current Occupation (PISA YAFS): The variable is 
derived from the respondent’s employment status, 

BQ Q8, and their occupation category, BQ Q9_ TOP, 
reported as four-digit codes of the International Standard 
Classification of Occupations, 2008 version (ISCO-08). 
When the respondent is employed, the values of the 
variable come directly from BQ _Q9 TOP. When the 
respondent is employed and BQ Q9 TOP is missing, the 
variable is coded as missing. When the respondent is 
unemployed and not looking for work, the variable is 
coded as unemployed (not looking for work). When the 
respondent is unemployed and looking for work, the 
variable is coded as unemployed (looking for work). 


The values for current occupation are as follows: 
1 = Armed forces occupations 

2 = Managers 

3 = Professionals 

4 = Technicians and associate professionals 

5 = Clerical support workers 

6 = Service and sales workers 

7 = Skilled agricultural, forestry, and fish 

8 = Craft and related trades workers 

9 = Plant and machine operators, and assembly 
91 = Elementary occupations 

92 = Unemployed (not looking for work) 

93 = Unemployed (looking for work) 

999 = Missing 


Level of job-seeking behavior (PISA YAFS): The 
variable is derived from the respondent’s job-seeking 
intentionality score, SEEKJOB_GROUP, and employment 
status, BQ Q8. When the respondent’s employment status 
is not unemployed (not seeking employment), the values 
of the variable come directly from SEEKJOB_GROUP. If the 
respondent’s employment status is not unemployed (not 
seeking employment) and SEEKJOB_GROUP is NA, the 
variable is coded as NA. If the respondent’s employment 
status is not unemployed (not seeking employment) 

and SEEKJOB_GROUP is missing, the variable is coded 

as missing. If the respondent’s employment status is 
unemployed (not seeking employment), the variable is 
coded as unemployed (not seeking employment). 


The values for level of job-seeking behavior are as 
follows: 

1= Low 

2 = Moderate 

3 = High 

4 = Unemployed (not seeking employment) 

998 = NA 

999 = Missing 


Level of self-efficacy (PISA YAFS): The variable 
is derived from the respondent’s job-seeking and 


training self-efficacy score, SELFEFFICACY_GROUP, and 


employment status, BQ_Q8. When the respondent’s 
employment status is not unemployed (not seeking 
employment), the values of the variable come directly 
from SELFEFFICACY GROUP. If the respondent’s 
employment status is not unemployed (not seeking 
employment) and SELFEFFICACY GROUP is NA, the 


variable is coded as NA. If the respondent’s employment 
status is not unemployed (not seeking employment) and 
SELFEFFICACY GROUP is missing, the variable is coded 


as missing. If the respondent’s employment status is 


unemployed (not seeking employment), the variable is 


coded as unemployed (not seeking employment). 


The values for level of self-efficacy are as follows: 
1= Low 

2 = Moderate 

3 = High 

4 = Unemployed (not seeking employment) 

998 = NA 

999 = Missing 


Satisfaction with life (PISA YAFS): The variable 
indicates the respondent’s satisfaction level with life. 
It comes directly from the satisfaction with life score, 
SATISFACTION _ LIFE LABEL. 


The values for satisfaction with life are as follows: 
1= Low 

2 = Moderate 

3 = High 

9 = Missing 
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Average on the vocational interests index (PISA 
YAFS): These six variables are the mean scores of six 
career intentionality variables. The Realistic is the mean 
estimate of PERSON_R_TOTAL using replicate weights. 
The Investigative is the mean estimate of PERSON_I 
TOTAL using replicate weights. The Artistic is the mean 
estimate of PERSON_A_TOTAL using replicate weights. 
The Social is the mean estimate of PERSON_S TOTAL 
using replicate weights. The Enterprising is the mean 
estimate of PERSON_E_TOTAL using replicate weights. 
The Conventional is the mean estimate of PERSON_C_ 
TOTAL using replicate weights. 


Appendix C. 
Technical Notes 


Assessment design 


PISA YAFS measures both cognitive and noncognitive 
constructs related to adult life preparedness, skill use, 
and achievement in young adulthood. The PISA YAFS 
data for 19-year-olds come from two instruments: the 
Education and Skills Online (ESO) assessment and 
questionnaire and an additional series of questions called 
the Learning Experience Questionnaire (LEQ), which 
were selected to supplement the ESO’s background 
questionnaire. As ESO did not have information on 
current educational status and could be administered 
only as designed, the LEQ module was developed to 
more fully realize the aims of PISA YAFS. In particular, 
the questionnaire items in the LEQ module allow 
examination of the paths that individuals choose after 
leaving high school. The questionnaire items in the LEQ 
were selected from questionnaire items in PIAAC that 
were not included in ESO. 


ESO includes a core set of background items, two 
cognitive assessment blocks, first being the router, and 
multiple noncognitive questionnaire modules. The 
noncognitive indicators collected through PISA YAFS 
include categorical and continuous data from the LEQ 
and ESO noncognitive (background questionnaire) 
modules. The LEQ gathered information on (1) current 
education study status (participation, level of degree, 
area of study), (2) formal education activities, and 

(3) nonformal learning activities in the 12 months 
preceding the study. The ESO noncognitive modules 
collected information on respondents’ (1) basic 
demographics, (2) career interests and intentionality 
(CII), (3) behavioral performance competencies (BPC), 
and (4) subjective well-being and health (SWBH). Each of 
these modules is represented in the PISA YAFS database 
by the questionnaire-item-level data and composite 
derived variables (e.g., life satisfaction index). 


As in PIAAC, the ESO cognitive assessments included 
an adaptive element that allowed for automatic scoring. 
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Based on respondents’ performance at different points in 
the assessment, respondents were directed to different 
“testlets” that contained items of different average 
difficulty in the domains of literacy and numeracy. 


Unlike PIAAC, ESO was self-administered and offered 
only via computer; there was no paper-and-pencil 
component. As in PIAAC, ESO task items were designed 
to be authentic, culturally appropriate, and drawn from 
real-life situations that are expected to be of importance 
or relevance in different contexts. Task item content and 
questions are intended to reflect the purposes of adults’ 
daily lives, even if they are not necessarily familiar to all 
adults. 


Sampling and response rates 


Respondents were contacted to participate in PISA YAFS 
if they voluntarily returned a Student Information Form 
in PISA 2012. The PISA 2012 core assessment had 4,978 
participating U.S. students. Of these, 4,612 students 
(92.6 percent) returned an information form (table C-1). 
A respondent in PISA YAFS was considered participating 
if they completed at least one LEQ item regardless 

of whether they went on to complete any of the ESO 
noncognitive or cognitive sections. 


Nonresponse bias 


The nonresponse bias analysis of the PISA YAFS sample 
revealed differences in the characteristics of respondents 
who participated in the PISA YAFS with the total eligible 
sample. The eligible sample are those who participated 
in PISA 2012 at age 15, but not necessarily in PISA YAFS 
at age 19. A portion of the eligible sample participated in 
PISA YAFS, referred to as participants: 


© Black or African American individuals were 


underrepresented among participants relative to the 
eligible sample (9.1 vs. 12.5 percent, respectively), 
while White individuals were overrepresented among 
participants (53.8 vs. 50.4 percent, respectively). 


Table C-1. | Number of PISA 2012 respondents participating in PISA YAFS 2016 main study 


Percentage Percentage 
(COTA eVCe Lal =xe)] Zee Lai=xe)} 
SSTUTaY(eNVam oy-Vad(ed| or-T al Number | response rate | response rate 
U.S. PISA 2012 student sample (core assessment only) 4,978 
PISA 2012 students completing Student Information Forms 4,612 100.0 100.0 
PISA YAFS 2016 participants 2,318 50.3 50.1 
Participated in LEQ 2,318 50.3 50.1 
Participated in ESO 2,008 43.5 43.6 
Completed literacy ESO assessment 1,823! 39.5 39.5 
Completed numeracy ESO assessment 1,854? 40.2 40.2 
4 Thirty-three of these respondents completed the assessment but failed to press the final “Submit” button. Their responses were recovered during data 
? Sixty-four of these respondents completed the assessment but failed to press the final “Submit” button. Their responses were recovered during data 
processing. 


NOTE: Tracking efforts that followed up with PISA 2012 respondents on the information provided in the information forms they filled out at the end of the 
PISA 2012 are not reflected in this table. Of the 4,612 PISA 2012 participants who returned the forms, 2,448 registered on the PISA YAFS recruitment 
website. LEQ refers to the additional noncognitive survey items in the Learning Experience Questionnaire. ESO refers to the online assessment known 
as Education and Skills Online. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, Program for International Student Assessment (PISA), 2012; and 
PISA Young Adult Follow-up Study (PISA YAFS), 2016 


© Male individuals were underrepresented among 


participants relative to the eligible sample (44.7 vs. 
51.0 percent, respectively), while female individuals 
were overrepresented among participants (55.3 vs. 
49.0 percent, respectively). 


Individuals who did not attend preschool or 
kindergarten were underrepresented among 
participants relative to the eligible sample (1.2 vs. 
1.5 percent, respectively), while individuals who 
attended preschool or kindergarten for more than 
1 year were overrepresented among participants 
(74.8 vs. 73.7 percent, respectively). 


Individuals who attended preschool for 1 year or 
less were underrepresented among participants 
relative to the eligible sample (43.4 vs. 45.7 percent, 
respectively), while individuals who attended for 
more than 1 year were overrepresented among 
participants (31.5 vs. 27.5 percent, respectively). 


Individuals who were truant one or two times were 
underrepresented among participants relative to the 
eligible sample (16.5 vs. 17.7 percent, respectively), 
while individuals who were never truant were 
overrepresented among participants (81.0 vs. 

78.1 percent, respectively). 


relative to the eligible sample (4.9 vs. 7.7 percent, 
respectively), while individuals who expected to 
complete a doctoral or professional degree were 
overrepresented among participants (27.0 vs. 
22.9 percent, respectively). 


Individuals with the highest level of parental 
educational attainment of ISCED level 3A or 4 were 
underrepresented among participants relative to the 
eligible sample (28.2 vs. 31.7 percent, respectively), 
while individuals with ISCED level 5A or 6 were 
overrepresented among participants (48.9 vs. 

43.2 percent, respectively). 


Individuals whose father’s current job status was 
working part-time with pay were underrepresented 
among participants relative to the eligible sample 
(5.7 vs. 6.4 percent, respectively), while individuals 
whose father was working full-time for pay were 
overrepresented among participants (76.0 vs. 

72.8 percent, respectively). 


Individuals had a higher mean of plausible value 1 in 
mathematics, reading, and science than the eligible 
sample (507.6 vs. 480.7, 525.2 vs. 497.0, and 524.8 vs. 
497.2, respectively) and a higher mean of openness for 
problem solving than the eligible sample (0.3 vs. 0.2). 


The analysis showed that weighting adjustments 
were highly effective in reducing the bias. Apart from 


© Individuals who expected to complete high school 
were underrepresented among participants 
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the couple of exceptions listed below, there were no 
statistically significant differences between participants 
and the eligible sample once nonresponse adjusted 
weights were applied 


© Individuals who attended preschool for 1 year or 
less were overrepresented among participants 
relative to the eligible sample (46.6 vs. 45.7 percent, 
respectively), while individuals who attended for 
more than 1 year were underrepresented among 
participants (26.6 vs. 27.5 percent, respectively). 


© Individuals who expected to complete high school 
were overrepresented among participants relative to 
the eligible sample (8.7 vs. 7.7 percent, respectively), 
while individuals who expected to complete a 
master’s degree were underrepresented among 
participants (21.6 vs. 22.3 percent, respectively). 


For both of these characteristics, the nonresponse 
adjustment slightly overadjusted for most of the 
categories (as seen by the change in the sign of the bias) 
but still greatly reduced the bias in all categories of 
“highest grade expected to complete” and all but one in 
“attended preschool.” 


More details and a description of the rescaling 
methodology and considerations are provided in 
Chapter 5 of the PISA YAFS Technical Report (Kastberg 
et al. forthcoming). 


Assessments and 


questionnaires 


The PISA Young Adult Follow-up Study (PISA YAFS), 
including the Education and Skills Online (ESO) 
assessment, was administered in 2016, which was 

34% years after the respondents had taken PISA 2012. 
From the end of the PISA assessment in fall 2012 through 
March 2016, respondents were traced using the contact 
information they provided after the 2012 assessment 
(meaning they were contacted via email, phone, or 
physical address) and tracked meaning that the study 
maintained communication channels with respondents 
all the way through data collection. During this period 
potential participants were asked to log into the system 
and update their contact information. 


As the data collection window approached, the study 
launched a recruitment effort utilizing the tracing and 
tracking results. Since the tracing and tracking stage did 
not include responding to any instruments or other tasks 
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along the way, as long as the participant was not marked 
as an active refusal and the study had working contact 
information, recruitment was attempted. 


The PISA YAFS field test conducted December 
2015-January 2016 with PISA 2012 financial literacy 
participants was focused primarily on proving that 
the design and operations for collecting the data were 
feasible and robust. 


The main study data collection for PISA YAFS occurred 
between March and July 2016. 


The PISA YAFS data collection instruments included 

the Learning Experience Questionnaire (LEQ) and the 
ESO assessment. The LEQ is a noncognitive background 
questionnaire about current learning experiences, based 
on components of the PIAAC background questionnaire 
that were not included in ESO. The items in the LEQ 
were added to supplement the ESO noncognitive 
measures. The ESO is a set of noncognitive and cognitive 
instruments that operate together to produce scale scores 
and noncognitive indicators (for more information, 

see http://www.oecd.org/skills/ESonline-assessment/ 
abouteducationskillsonline/). 


The ESO platform required users to use a single specific 
browser, Firefox, and access the assessment from a 
laptop or PC (i.e., no tablets or cellphones). The LEQ did 
not have any device-type or browser restriction. The LEQ 
was administered prior to the ESO. 


More details and a description of the survey operations 
and questionnaires are provided in Chapter 3 of the PISA 
YAFS Technical Report (Kastberg et al. forthcoming). 


Reporting results 


The assessment tool used in PISA YAFS in 2016 was the 
ESO survey. The ESO was developed to provide individual 
results, by way of calculating and presenting a score and 
a proficiency level for each participant who completes 
the literacy and numeracy assessments. While the scores 
produced by the ESO are designed to place respondents’ 
achievement on the same scale as PIAAC, ESO scores 

are calculated in increments of 10 (e.g., 350, 360, 370) 
and do not include standard errors that account for 
measurement error. In order to improve the utility and 
statistical power of the ESO cognitive data for reporting, 
the ESO scale scores were transformed into plausible 
values using item response theory (IRT), which provides 
more accurate measurement of student performance 
than the ESO scale scores. More details and a description 


of the rescaling methodology and considerations are 
provided in Chapter 6 of the PISA YAFS Technical Report 
(Kastberg et al. forthcoming). 


Sampling weights and 
standard errors 


Generally, in a survey, the base weights are computed 
from the inverse of the probability of selection for the 
sample. However, since PISA YAFS is a follow-up to 

PISA 2012, the base weights for PISA YAFS are the final 
PISA 2012 student weights. These PISA 2012 student 
weights were then adjusted to compensate for PISA YAFS 
nonresponse. There were two sources of nonresponse: 
PISA students who did not provide follow-up contact 
information in 2012 during the PISA assessment and 
those who did not respond to the PISA YAFS survey. 


Statistical comparisons 


Two types of statistical comparisons were used for 
this report. One tested observed differences between 
percentages and between means, and another tested 
observed differences between correlations, based on 
the t statistic. 


For the percentages and means, statistical significance 
was determined by calculating a t value for the difference 
between a pair of means or proportions, comparing 
with published tables of values at a certain level of 
significance, called the alpha level. The alpha level is an 
a priori statement of the probability of inferring that a 
difference exists when, in fact, it does not. In this report, 
findings from tf tests are reported based on a statistical 
significance (or alpha level) set at .05. Respondents’ 

t values were computed to test differences between 
independent estimates using the following formula: 


E,-E, 
2 2 
{se?+ se; 


where E, and E, are the estimates to be compared and 
se, and se, are their corresponding standard errors. In 
instances where comparisons were made on dependent 
samples, the test statistic calculation was adjusted for 
the shared variance in the dependent groups using the 
following formula: 


a E,~E, 
{se2+ se? + 28e,se,” 


No adjustments were made for multiple comparisons. 
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To compare the two correlation coefficients, both 
correlation coefficients were converted to Fisher z values, 
computing the difference between the two z values. Then 
a “critical value” x is computed for the two estimates 
where the sample sizes are different using this formula: 


3 (n,- 3)(n,- 3) 
ie ca ee 


where q_ represents a difference between the two z values 
and n, and n, are the sample size for the two estimates, 
respectively. Finally, similar to measurement of statistical 
significance for percentages and means, the critical value 
x is compared with published tables of values at the alpha 
level set at .05. Here, too, no adjustments were made for 
multiple comparisons. 


t= 


There are some potential cautions in interpreting the 
results of statistical tests. First, the magnitude of the 

t statistic depends not only on observed differences 
between means or percentages but also on the number 
of respondents. A small difference found in a comparison 
across a large number of respondents would still produce 
a large and possibly statistically significant t statistic. 


A second caution stems from reliance on a sample, 
rather than an entire population: One can conclude 
that a difference found in the sample is real when there 
is no true difference in the population. Statistical tests 
are designed to limit the risk of this Type 1, or “false 
positive,” error by setting a significance level, or alpha. 
The alpha level of .05 used in this report ensures that the 
probability of finding a false positive result is no more 
than 1 in 20 (.05) occurrences. However, failing to meet 
the significance level of .05 does not mean that there is 
no real difference between two quantities, only that the 
likelihood is less. 


It is important to note that many of the variables 
examined in this report may be related to one another 
and to other variables not included in the analysis. 

The complex interactions and relationships among the 
variables were not explored. Furthermore, the variables 
examined in this report are just a few of those that could 
be examined. Thus, readers are cautioned not to draw 
causal inferences based on the results presented here. 


